-- Logs begin at Tue 2022-09-27 16:41:40 UTC, end at Tue 2022-09-27 17:49:31 UTC. -- Sep 27 17:09:47.548445 dv-cinder-849143-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 27 17:09:47.555714 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 27 17:09:47 2022] *** Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: nodename: dv-cinder-849143-8-iscsi Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: machine: x86_64 Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: clock source: unix Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: pcre jit disabled Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: detected number of CPU cores: 4 Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: current working directory: / Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: detected binary path: /usr/bin/uwsgi-core Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: your processes number limit is 47680 Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: your memory page size is 4096 bytes Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: detected max file descriptor number: 2048 Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: lock engine: pthread robust mutexes Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: thunder lock: enabled Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 27 17:09:47.560679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 27 17:09:47.598679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: Python main interpreter initialized at 0x55fb16c639c0 Sep 27 17:09:47.598679 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: python threads support enabled Sep 27 17:09:47.598900 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: your server socket listen backlog is limited to 100 connections Sep 27 17:09:47.598900 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: your mercy for graceful operations on workers is 90 seconds Sep 27 17:09:47.599056 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: mapped 403077 bytes (393 KB) for 2 cores Sep 27 17:09:47.599137 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: *** Operational MODE: preforking *** Sep 27 17:09:47.599387 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: *** uWSGI is running in multiple interpreter mode *** Sep 27 17:09:47.599387 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: spawned uWSGI master process (pid: 81415) Sep 27 17:09:47.599584 dv-cinder-849143-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 27 17:09:47.600133 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: spawned uWSGI worker 1 (pid: 81425, cores: 1) Sep 27 17:09:47.600461 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: spawned uWSGI worker 2 (pid: 81426, cores: 1) Sep 27 17:09:47.600611 dv-cinder-849143-8-iscsi devstack@keystone.service[81415]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 27 17:09:51.865540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 17:09:51.865540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: from cryptography.utils import int_from_bytes Sep 27 17:09:51.866342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 17:09:51.866342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: from cryptography.utils import int_from_bytes Sep 27 17:09:51.866758 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 17:09:51.866758 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: from cryptography.utils import int_from_bytes Sep 27 17:09:51.866866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 17:09:51.866866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: from cryptography.utils import int_from_bytes Sep 27 17:09:51.915428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 27 17:09:51.915639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 27 17:09:51.915831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] command line args: [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 27 17:09:51.916099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 27 17:09:51.916276 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ================================================================================ {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 27 17:09:51.916354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 27 17:09:51.916494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 27 17:09:51.916583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] admin_token = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.916692 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] command line args: [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 27 17:09:51.916777 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] config_dir = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.916927 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.917004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 27 17:09:51.917112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] config_source = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.917189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ================================================================================ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 27 17:09:51.917288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] debug = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.917367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] admin_token = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.917484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.917741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] config_dir = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.917741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.917857 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.917913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] config_source = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.918044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.918175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] debug = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.918288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.918384 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.918509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.918614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.918713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.918814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.918913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_config_append = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.919013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.919113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.919206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.919298 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_dir = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.919401 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.919501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_file = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.919607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_config_append = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.919701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.919807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.919892 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.920020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_dir = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.920020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_file = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.920197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.920277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.920392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.920476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.920598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.920672 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.920795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.920919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.921003 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.921085 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.921209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.921284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.921397 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.921517 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.921614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.921738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.921822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.921965 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.922043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.922219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.922305 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.922450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.922527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.922680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.922766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.922926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.923169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.923300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.923377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] publish_errors = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.923496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.923578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.923802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.923914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.924040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.924115 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] publish_errors = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.924239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.924320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.924426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.924531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.924660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] standard_threads = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.924746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.924892 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.924975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.925131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.925209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.925308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.925385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] standard_threads = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.925501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] use_journal = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.925581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.925653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] use_json = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.925807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] use_stderr = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.925953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.926034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.926127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] use_syslog = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.926209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_journal = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.926282 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.926350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_json = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.926469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.926548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_stderr = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.926653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_syslog = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.926727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.926881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 17:09:51.926957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.927137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.927215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.927382 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.927465 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.927639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.927718 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.927877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.927955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.928110 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.928191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.928347 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.928424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.928589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.928666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.928822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.928903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.929057 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.929139 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.929300 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.929378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.929532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.929609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.929765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.929840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.930009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.930116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.930270 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.930348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.930515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.930593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.930766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.930842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.930999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.931600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.931775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.931933 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.932013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.932085 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.932266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.932356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.932502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.932589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.932751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.932833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.932975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.933051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.933198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.933284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.933428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.933505 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.933662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.933739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.933912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.933997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.934162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.934252 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.934398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.934482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.934625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.934709 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.934870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.934947 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.935095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.935177 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.935325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.935410 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.935567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.935642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.935797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.935879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.936032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.936117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.936264 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.936346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.936493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.936575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.936718 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.936803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.936946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.937023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.937171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.937246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.937400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.937547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.937621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.937691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.937844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.937922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.938091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.938184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.938331 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.938408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.938566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.938684 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.938796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.938892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.939022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.939113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.939256 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.939335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.939487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.939575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.939722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.939865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.939995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.940127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.940262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.940388 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.940512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.940637 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.940791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.940877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.940957 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.941089 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.941168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.941275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.941352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.941423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.941546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.941621 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.941721 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.941808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.941957 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.942036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.942209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.942366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.942454 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.942543 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.942672 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.942748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.942903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.942987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.943138 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.943217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.943372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.943452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.943614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.943741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.943874 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.943994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.944136 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.944258 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.944380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.944500 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.944632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.944753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.944875 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.944996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.945128 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.945248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.945379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.945502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.945628 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.945754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.945883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.946015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.946211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.946333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.946460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.946583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.946717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.947443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.947579 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.947708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.947835 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.947955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.948038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.948111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.948111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.948249 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.948326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.948397 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.948397 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.948523 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.948604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.948676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.948745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.948823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.948957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.949035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.949182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.949256 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.949414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.949497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.949643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.949733 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.949873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.949960 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.950120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.950202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.950351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.950428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.950578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.950669 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.950804 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.950893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.951037 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.951120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.951268 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.951356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.951504 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.951581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.951737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.951819 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.951983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.952070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.952217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.952295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.952444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.external = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.952521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.952672 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.952760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.952939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.953024 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.953170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.external = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.953255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.953396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.password = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.953473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.953623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.token = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.953714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.953860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.953939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.954111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.954201 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.password = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.954357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.954444 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.token = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.954586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.954666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.954829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.954915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.955072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.955161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.955299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.955377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.955535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.955613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.955777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.955863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.956007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.956096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.956243 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.956321 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.956483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.956576 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.956727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.956805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.956961 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.957039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.957206 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.957293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.957446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.957523 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.957683 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.957762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.957929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.958019 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.958202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.958283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.958441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.958519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.958693 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.958784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.958944 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.959021 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.959198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.959276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.959435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.959515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.959670 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.959749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.959914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.959992 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.960151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.960229 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.960378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.960454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.960620 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.960698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.960855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.960934 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.961084 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.961161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.961331 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.961408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.961565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.961644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.961807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.961884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.962041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.962133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.962301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.962380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.962569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.962654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.962817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.962896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.963040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.963154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.963317 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.963403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.963566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.963645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.963798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.963878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.964033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.964114 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.964278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.964355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.964509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.964587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.964745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.964822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.964975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.965053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.965205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.965285 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.965445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.965522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.965677 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.965755 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.965908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.965992 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.966182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.966265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.966428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.966501 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.966673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.966752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.966920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.966997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.967157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.967235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.967389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.967468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.967629 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.967706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.967858 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.967944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.968093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.968171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.968325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.968402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.968556 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.968642 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.968793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.968871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.969028 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.969105 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.969265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.969350 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.969508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.969586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.969735 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.969813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.969979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.970064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.970249 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.970329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.970479 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.970557 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.970715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.970805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.970945 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.971024 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.971174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.971252 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.971408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.971493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.971634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.971712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.971871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.971949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.972105 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.972218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.972346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.972471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.972582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.972721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.972931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.973018 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.973172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.973260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.973421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.973501 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.973652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.973737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.973882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.973961 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.974142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.974222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.974384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.974473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.974616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.974695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.974851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.974928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.975089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.975175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.975321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.975400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.975561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.975640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.975806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.975894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.976053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.976131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.976290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.976375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.976523 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.976610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.976760 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.976853 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.977008 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.977109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.977227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.977318 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.977456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.977533 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.977690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.977767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.977926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.978020 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.978200 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.978283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.978449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.978527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.978694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.978785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.978926 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.979004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.979171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.979248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.979405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.979495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.979632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.979710 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.979864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.979941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.980100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.980177 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.980339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.980450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.980575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.980660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.980809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.980902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.981052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.981132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.981281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.981376 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.981529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.981633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.981763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.981846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.981995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.982108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.982255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.982350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.982489 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.982569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.982724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.982805 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.982953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.983041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.983190 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.983271 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.983427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.983507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.983659 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.983741 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.983897 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.983978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.984129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.984208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.984377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.984459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.984623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.984703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.985119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.985206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.985366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.985448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.985602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.985683 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.985840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.985921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.986093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] role.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.986173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.986330 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.986410 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] role.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.986559 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.986638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.986794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] role.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.986875 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.987041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] role.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.987123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.987271 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.987352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.987519 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.987602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.987763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.987844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.988008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.988087 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.988251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.988343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.988493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.988576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.988722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.988802 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.988968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.989050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.989212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.989293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.989440 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.989519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.989684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.989767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.989929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.990094 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.990200 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.990337 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.990449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.990585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.990833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.990920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.991066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.991154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.991314 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.991393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.991547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.991627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.991793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.991881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.992041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.992120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.992271 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.992352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.992500 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.992588 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.992731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.992813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.992962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.993043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.993202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.993291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.993434 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.993515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.994216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.994301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.994517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.994618 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.994722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.994802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.994965 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.995050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.995200 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.995288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.995437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.995518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.995664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.995744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.995900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.995989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.996136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.996217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.996382 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.996469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.996617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.996714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.996860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.996941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.997104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.997184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.997337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.997424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.997573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.997654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.997812 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.997892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.998051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.998169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.998309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.998389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.998549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.998630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.998783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.998876 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.999027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.999116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.999293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.999374 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.999521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.999609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.connection = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.999773 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:51.999855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.000016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.000098 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.000254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.000335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.000495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.000575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.000730 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.000812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.000966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.001048 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.001200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.001280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.001432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.001513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.001668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.001749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.001900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.001979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.002152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.002237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.002389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.002468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.002621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.002700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.002856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.002938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.003096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.003175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.003323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.003403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.003564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.003646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.003798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.003880 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.003954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 27 17:09:52.004036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.004252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.004488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 17:09:52.004658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 27 17:09:52.033323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.033555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.033783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.034186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.034420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.042715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.043615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.070356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.070584 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.071577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.071969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.124837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.124971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.173083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.173318 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.173909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.174176 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.328052 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.328284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.328383 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.328508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.328654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.328742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.328820 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.328925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.329011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.329138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.329223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.329382 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.352353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.352584 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.352806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.353656 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.353741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.353943 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.354024 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.354196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.354390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.354521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.354643 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.354763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.354882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.355008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.366316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.366702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.379506 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.379735 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.379895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.380031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.380165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.380296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.380427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.380557 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.380694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.380823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.380953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.381082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.381082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.381352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 27 17:09:52.397264 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.397474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 27 17:09:52.397816 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.397946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.398122 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 27 17:09:52.398262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 27 17:09:52.398397 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.398529 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.398660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.398803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 27 17:09:52.398945 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.399096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.399226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.399355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.399482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.399673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.408597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.408877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.409149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.409307 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.409441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.409592 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.409847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.410164 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.416900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.417646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.419818 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.420104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.420383 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.420576 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.420707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.420851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.420995 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.421126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.421260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.421390 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.421522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.421659 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.421792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.421920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.422210 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.422356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.422472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.422559 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.422747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.423022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.423295 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.445538 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.446472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.447058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.447342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.447990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.448277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.452001 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.452283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.452564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.452935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.453220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.453499 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.458876 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.459164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.459839 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.460128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.464780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.465070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.465354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.465637 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.465769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.465912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.466042 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.466201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.466332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.466464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.466617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.466751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.466885 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.467158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.467438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.467712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.482580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.482858 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.483619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.483902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.486978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.488050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.490219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.490516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.491301 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.491600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.495276 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.496360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.498386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.499561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.501579 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.501879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.502186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.502474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.502756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.502885 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.503053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.503186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.503360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.503646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.503923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.504202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.518683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.518973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.519241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.519502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.519905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.520197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.520469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.520739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.526335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.527632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.527879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.529170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.529404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.529680 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.530743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.531023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.532351 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.532633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.532916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.533205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.533607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.533781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.534094 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.534370 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.534665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.535064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.545218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.545508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.545792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.546101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.546543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.546738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.546872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.547028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.547161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.547315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.547607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.547990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.548288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.548570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.562379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.563953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.565284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.566894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.568161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.569613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.569792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.571117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.571253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.571407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.572337 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.573338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.573622 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.574018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.578685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.580147 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.580367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.581821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.583040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.583439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.583734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.584015 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.584741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.585144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.585436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.585718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.591831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.592125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.592408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.593586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.593881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.594190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.599273 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.599557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.599838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.600117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.600401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.600675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.600949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.601092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 27 17:09:52.601227 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.601360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.601493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.601626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.601765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.601915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.602113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.602247 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.602387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.602517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.602648 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.602777 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.603443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.603619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.603751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.604000 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.604278 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.604550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.604816 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.605095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 27 17:09:52.638891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55fb16c639c0 pid: 81426 (default app) Sep 27 17:09:52.639860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55fb16c639c0 pid: 81425 (default app) Sep 27 17:09:52.642783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e7e8b-c496-49f0-86e0-d29b03d0c351 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:09:52.643069 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e7e8b-c496-49f0-86e0-d29b03d0c351 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:09:52.643342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e7e8b-c496-49f0-86e0-d29b03d0c351 None None] PATH_INFO: `/v3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:09:52.647525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1/1] 10.222.0.55 () {50 vars in 791 bytes} [Tue Sep 27 17:09:52 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:09:59.433680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b752e-adca-4892-9ccc-fe8d3e6cac54 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:09:59.434215 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b752e-adca-4892-9ccc-fe8d3e6cac54 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:09:59.434303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b752e-adca-4892-9ccc-fe8d3e6cac54 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:09:59.438591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1/2] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:09:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:09:59.443700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78c4d3-4b12-4174-8b95-84e9aa6d8f2b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:09:59.444189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78c4d3-4b12-4174-8b95-84e9aa6d8f2b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:09:59.444281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78c4d3-4b12-4174-8b95-84e9aa6d8f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:09:59.514659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: INFO dbcounter [None req-eb78c4d3-4b12-4174-8b95-84e9aa6d8f2b None None] Registered counter for database keystone Sep 27 17:09:59.533150 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG oslo_db.sqlalchemy.engines [None req-eb78c4d3-4b12-4174-8b95-84e9aa6d8f2b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81426) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 17:09:59.539268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writer thread running {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 17:09:59.713537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG passlib.handlers.bcrypt [None req-eb78c4d3-4b12-4174-8b95-84e9aa6d8f2b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81426) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 27 17:09:59.713914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG passlib.handlers.bcrypt [None req-eb78c4d3-4b12-4174-8b95-84e9aa6d8f2b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81426) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 27 17:09:59.771683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-eb78c4d3-4b12-4174-8b95-84e9aa6d8f2b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:09:59.773280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-eb78c4d3-4b12-4174-8b95-84e9aa6d8f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:09:59.896893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 2/3] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:09:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 27 17:10:00.205500 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b752e-adca-4892-9ccc-fe8d3e6cac54 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:00.275299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: INFO dbcounter [None req-2f2b752e-adca-4892-9ccc-fe8d3e6cac54 None None] Registered counter for database keystone Sep 27 17:10:00.296082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG oslo_db.sqlalchemy.engines [None req-2f2b752e-adca-4892-9ccc-fe8d3e6cac54 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81425) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 17:10:00.302180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writer thread running {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 17:10:00.421318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b752e-adca-4892-9ccc-fe8d3e6cac54 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:00.425581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102e45b-9b9f-463e-b5dd-97ba602ebb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:00.426224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102e45b-9b9f-463e-b5dd-97ba602ebb50 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:00.426506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102e45b-9b9f-463e-b5dd-97ba602ebb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:00.426777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102e45b-9b9f-463e-b5dd-97ba602ebb50 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:00.435664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102e45b-9b9f-463e-b5dd-97ba602ebb50 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:00.486896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f102e45b-9b9f-463e-b5dd-97ba602ebb50 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 27 17:10:00.487187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102e45b-9b9f-463e-b5dd-97ba602ebb50 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:00.496981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 2/4] 10.222.0.55 () {58 vars in 1324 bytes} [Tue Sep 27 17:10:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 17:10:00.502913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78c4d3-4b12-4174-8b95-84e9aa6d8f2b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:00.519946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78c4d3-4b12-4174-8b95-84e9aa6d8f2b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:00.524302 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6cca4-5b7f-422d-8c0b-a9493a9a527a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:00.525407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6cca4-5b7f-422d-8c0b-a9493a9a527a admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:00.525685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6cca4-5b7f-422d-8c0b-a9493a9a527a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:00.525951 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6cca4-5b7f-422d-8c0b-a9493a9a527a admin admin] PATH_INFO: `/v3/projects/5c93f41c1d2243bfbbfb547f2005e999` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:00.527554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6cca4-5b7f-422d-8c0b-a9493a9a527a admin admin] RBAC: Authorizing `identity:get_project(project_id=5c93f41c1d2243bfbbfb547f2005e999)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:00.580570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dfd6cca4-5b7f-422d-8c0b-a9493a9a527a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 27 17:10:00.580913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd6cca4-5b7f-422d-8c0b-a9493a9a527a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:00.585346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 3/5] 10.222.0.55 () {56 vars in 1207 bytes} [Tue Sep 27 17:10:00 2022] GET /identity/v3/projects/5c93f41c1d2243bfbbfb547f2005e999 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:02.733097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a5633-560a-431b-baaa-9843d68963eb None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:02.733702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a5633-560a-431b-baaa-9843d68963eb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:02.733702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a5633-560a-431b-baaa-9843d68963eb None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:02.738021 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 3/6] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:10:02.744268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc04c26-a5c9-4a8d-a437-077169c91ef9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:02.744682 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc04c26-a5c9-4a8d-a437-077169c91ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:02.744847 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc04c26-a5c9-4a8d-a437-077169c91ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:02.765111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-acc04c26-a5c9-4a8d-a437-077169c91ef9 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:02.766666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-acc04c26-a5c9-4a8d-a437-077169c91ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:02.774478 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 4/7] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 27 17:10:03.049750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a5633-560a-431b-baaa-9843d68963eb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:03.062544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a5633-560a-431b-baaa-9843d68963eb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:03.066901 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d821e0-f23c-4607-920f-4dc1566b8038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:03.067489 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d821e0-f23c-4607-920f-4dc1566b8038 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:03.067763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d821e0-f23c-4607-920f-4dc1566b8038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:03.068029 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d821e0-f23c-4607-920f-4dc1566b8038 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:03.075619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d821e0-f23c-4607-920f-4dc1566b8038 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:03.077377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d821e0-f23c-4607-920f-4dc1566b8038 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:03.085853 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 4/8] 10.222.0.55 () {58 vars in 1324 bytes} [Tue Sep 27 17:10:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:10:03.091306 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc04c26-a5c9-4a8d-a437-077169c91ef9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:03.100271 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc04c26-a5c9-4a8d-a437-077169c91ef9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:03.105081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb2ca2-0964-4bc2-9e11-d333c8d7ac87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:03.105700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb2ca2-0964-4bc2-9e11-d333c8d7ac87 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:03.105980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb2ca2-0964-4bc2-9e11-d333c8d7ac87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:03.106286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb2ca2-0964-4bc2-9e11-d333c8d7ac87 admin admin] PATH_INFO: `/v3/users/60f027a86a574cfdbd746a543980fbdf` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:03.107585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb2ca2-0964-4bc2-9e11-d333c8d7ac87 admin admin] RBAC: Authorizing `identity:get_user(user_id=60f027a86a574cfdbd746a543980fbdf)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:03.109691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb2ca2-0964-4bc2-9e11-d333c8d7ac87 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:03.111247 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 5/9] 10.222.0.55 () {56 vars in 1198 bytes} [Tue Sep 27 17:10:03 2022] GET /identity/v3/users/60f027a86a574cfdbd746a543980fbdf => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:05.309711 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011262f-2576-442c-ac3c-5d72d8c9d403 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:05.310293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011262f-2576-442c-ac3c-5d72d8c9d403 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:05.310293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011262f-2576-442c-ac3c-5d72d8c9d403 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:05.311608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 5/10] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:05.316890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f585d8-83be-4834-b79e-e808091b846e None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:05.317292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f585d8-83be-4834-b79e-e808091b846e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:05.317468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f585d8-83be-4834-b79e-e808091b846e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:05.335625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c5f585d8-83be-4834-b79e-e808091b846e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:05.337398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c5f585d8-83be-4834-b79e-e808091b846e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:05.371366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02947366-1438-4179-89f6-d4c11d83489e None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:05.371647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02947366-1438-4179-89f6-d4c11d83489e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:05.371920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02947366-1438-4179-89f6-d4c11d83489e None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:05.373227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 6/11] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:10:05.378111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b277a8-83c9-4fa1-9003-97cf4f8e07d1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:05.378396 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b277a8-83c9-4fa1-9003-97cf4f8e07d1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:05.378669 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b277a8-83c9-4fa1-9003-97cf4f8e07d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:05.420788 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG passlib.handlers.bcrypt [None req-a4b277a8-83c9-4fa1-9003-97cf4f8e07d1 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81425) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 27 17:10:05.421171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG passlib.handlers.bcrypt [None req-a4b277a8-83c9-4fa1-9003-97cf4f8e07d1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81425) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 27 17:10:05.427808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 6/12] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:05.471234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a4b277a8-83c9-4fa1-9003-97cf4f8e07d1 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:05.472822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a4b277a8-83c9-4fa1-9003-97cf4f8e07d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:05.569211 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 7/13] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:05.701044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f585d8-83be-4834-b79e-e808091b846e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:05.726353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:05.804661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f585d8-83be-4834-b79e-e808091b846e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:05.836650 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b277a8-83c9-4fa1-9003-97cf4f8e07d1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:05.837414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7622518-a91a-4c4e-8a6d-3c4609ddee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:05.838049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7622518-a91a-4c4e-8a6d-3c4609ddee2a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:05.838353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7622518-a91a-4c4e-8a6d-3c4609ddee2a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:05.838629 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7622518-a91a-4c4e-8a6d-3c4609ddee2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:05.849649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7622518-a91a-4c4e-8a6d-3c4609ddee2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-c7622518-a91a-4c4e-8a6d-3c4609ddee2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:05.859049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:05.860240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:05.861208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:05.861208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:05.861208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:05.861208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:05.861208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:05.861208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:05.861208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:05.861208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:05.861208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:05.861208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:05.861208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:05.861208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:05.861208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 7/14] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:10:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:05.866119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7622518-a91a-4c4e-8a6d-3c4609ddee2a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:05.937138 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b277a8-83c9-4fa1-9003-97cf4f8e07d1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:05.939432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:05.947175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:05.965752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7622518-a91a-4c4e-8a6d-3c4609ddee2a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:05.973483 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a45d9-68ab-4267-b35e-3f229e27ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:05.974366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a45d9-68ab-4267-b35e-3f229e27ce5f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:05.974647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a45d9-68ab-4267-b35e-3f229e27ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:05.974919 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a45d9-68ab-4267-b35e-3f229e27ce5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:06.000874 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a65c0-8d5d-42c3-9408-e97ecdd7e4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:06.001547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a65c0-8d5d-42c3-9408-e97ecdd7e4ec None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:06.001820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a65c0-8d5d-42c3-9408-e97ecdd7e4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:06.002099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a65c0-8d5d-42c3-9408-e97ecdd7e4ec None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:06.002768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a65c0-8d5d-42c3-9408-e97ecdd7e4ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:06.003059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a65c0-8d5d-42c3-9408-e97ecdd7e4ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:06.004437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327a65c0-8d5d-42c3-9408-e97ecdd7e4ec None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:06.015964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 8/15] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:06.021351 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a65c0-8d5d-42c3-9408-e97ecdd7e4ec None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:06.043169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a45d9-68ab-4267-b35e-3f229e27ce5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:06.044682 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a45d9-68ab-4267-b35e-3f229e27ce5f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:06.120143 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a65c0-8d5d-42c3-9408-e97ecdd7e4ec None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:06.140783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 8/16] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 27 17:10:06.146318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a45d9-68ab-4267-b35e-3f229e27ce5f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:06.149513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:06.154590 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44f5c9-f0a2-450d-a853-a137bbbd28bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:06.155223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44f5c9-f0a2-450d-a853-a137bbbd28bb None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:06.155503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44f5c9-f0a2-450d-a853-a137bbbd28bb None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:06.155771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44f5c9-f0a2-450d-a853-a137bbbd28bb None admin] PATH_INFO: `/v3/users/60f027a86a574cfdbd746a543980fbdf` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:06.156925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44f5c9-f0a2-450d-a853-a137bbbd28bb None admin] RBAC: Authorizing `identity:get_user(user_id=60f027a86a574cfdbd746a543980fbdf)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:06.158410 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44f5c9-f0a2-450d-a853-a137bbbd28bb None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:06.159960 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 9/17] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:06 2022] GET /identity/v3/users/60f027a86a574cfdbd746a543980fbdf => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:06.165175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44f5c9-f0a2-450d-a853-a137bbbd28bb None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:06.179303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:06.243763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a45d9-68ab-4267-b35e-3f229e27ce5f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:06.263831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44f5c9-f0a2-450d-a853-a137bbbd28bb None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:06.278012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0ddbe-d9b3-4787-ad32-2aa8ef14262b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:06.278871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0ddbe-d9b3-4787-ad32-2aa8ef14262b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:06.279148 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0ddbe-d9b3-4787-ad32-2aa8ef14262b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:06.279420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0ddbe-d9b3-4787-ad32-2aa8ef14262b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:06.296947 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734808b-08fa-4405-be33-7edded2cf2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:06.297571 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734808b-08fa-4405-be33-7edded2cf2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:06.297859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734808b-08fa-4405-be33-7edded2cf2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:06.297986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0ddbe-d9b3-4787-ad32-2aa8ef14262b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:06.298153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734808b-08fa-4405-be33-7edded2cf2b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:06.299269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734808b-08fa-4405-be33-7edded2cf2b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:06.299381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0ddbe-d9b3-4787-ad32-2aa8ef14262b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:06.300714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734808b-08fa-4405-be33-7edded2cf2b9 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:06.302349 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 10/18] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:06.308121 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734808b-08fa-4405-be33-7edded2cf2b9 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-6ee0ddbe-d9b3-4787-ad32-2aa8ef14262b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:10:06.320175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:06.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:10:06.322249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:10:06.322249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:10:06.322249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:06.322249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 9/19] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:10:06.327286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0ddbe-d9b3-4787-ad32-2aa8ef14262b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:06.387262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:06.388641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:06.408200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734808b-08fa-4405-be33-7edded2cf2b9 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:06.433079 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0ddbe-d9b3-4787-ad32-2aa8ef14262b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:06.440889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98c6e3-d802-4f2e-affa-0ab5e034ab2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:06.441490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98c6e3-d802-4f2e-affa-0ab5e034ab2b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:06.441780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98c6e3-d802-4f2e-affa-0ab5e034ab2b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:06.442054 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98c6e3-d802-4f2e-affa-0ab5e034ab2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:06.442809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98c6e3-d802-4f2e-affa-0ab5e034ab2b None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.domain.id=default, user.id=60f027a86a574cfdbd746a543980fbdf) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:06.443115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98c6e3-d802-4f2e-affa-0ab5e034ab2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:06.444552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b98c6e3-d802-4f2e-affa-0ab5e034ab2b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:06.451034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 11/20] 10.222.0.55 () {56 vars in 1322 bytes} [Tue Sep 27 17:10:06 2022] GET /identity/v3/role_assignments?user.id=60f027a86a574cfdbd746a543980fbdf&scope.domain.id=default&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:06.467170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04856e-0806-4d31-8d15-7f6c2fc43636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:06.467803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04856e-0806-4d31-8d15-7f6c2fc43636 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:06.468082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04856e-0806-4d31-8d15-7f6c2fc43636 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:06.468352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04856e-0806-4d31-8d15-7f6c2fc43636 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:06.469022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04856e-0806-4d31-8d15-7f6c2fc43636 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:06.469325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04856e-0806-4d31-8d15-7f6c2fc43636 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:06.470807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04856e-0806-4d31-8d15-7f6c2fc43636 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:06.491286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 10/21] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:09.006858 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025aeb4e-75e1-4c9f-ab6a-884f4b248013 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:09.007385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025aeb4e-75e1-4c9f-ab6a-884f4b248013 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:09.007385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025aeb4e-75e1-4c9f-ab6a-884f4b248013 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:09.009322 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 12/22] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:09.014663 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706dcc4-05b9-4de8-a84f-3c7bc3fe1037 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:09.015724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706dcc4-05b9-4de8-a84f-3c7bc3fe1037 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:09.016189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706dcc4-05b9-4de8-a84f-3c7bc3fe1037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:09.044136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1706dcc4-05b9-4de8-a84f-3c7bc3fe1037 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:09.045813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1706dcc4-05b9-4de8-a84f-3c7bc3fe1037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:09.140226 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 11/23] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:09.186510 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec9820-8736-4040-baaa-d9a84b1b831a None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:09.187150 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec9820-8736-4040-baaa-d9a84b1b831a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:09.187633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec9820-8736-4040-baaa-d9a84b1b831a None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:09.189236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 13/24] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:09.194753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dabfa9-69b4-453e-ae5c-d2d103e7b7e8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:09.195322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dabfa9-69b4-453e-ae5c-d2d103e7b7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:09.195811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dabfa9-69b4-453e-ae5c-d2d103e7b7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:09.214719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b4dabfa9-69b4-453e-ae5c-d2d103e7b7e8 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:09.216206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b4dabfa9-69b4-453e-ae5c-d2d103e7b7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:09.253807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf27c0c-de0c-40d0-9eba-7e102588bf6d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:09.254427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf27c0c-de0c-40d0-9eba-7e102588bf6d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:09.254919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf27c0c-de0c-40d0-9eba-7e102588bf6d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:09.256701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 14/25] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:09.263040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f527ad4-47b8-4715-a7af-f7dfd33c88af None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:09.263567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f527ad4-47b8-4715-a7af-f7dfd33c88af None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:09.264030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f527ad4-47b8-4715-a7af-f7dfd33c88af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:09.294438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:09.297467 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9f527ad4-47b8-4715-a7af-f7dfd33c88af None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:09.299970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9f527ad4-47b8-4715-a7af-f7dfd33c88af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:09.350810 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 12/26] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:09.425669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 15/27] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:09.522396 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dabfa9-69b4-453e-ae5c-d2d103e7b7e8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:09.639937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dabfa9-69b4-453e-ae5c-d2d103e7b7e8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:09.679860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd067a1-93e5-401d-9095-a37b6f7f4a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:09.680745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd067a1-93e5-401d-9095-a37b6f7f4a81 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:09.681439 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd067a1-93e5-401d-9095-a37b6f7f4a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:09.681969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd067a1-93e5-401d-9095-a37b6f7f4a81 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:09.682843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd067a1-93e5-401d-9095-a37b6f7f4a81 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:09.684409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd067a1-93e5-401d-9095-a37b6f7f4a81 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:09.732648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f527ad4-47b8-4715-a7af-f7dfd33c88af None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:09.834472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 13/28] 10.222.0.55 () {60 vars in 1131 bytes} [Tue Sep 27 17:10:09 2022] POST /identity/v3/roles => generated 219 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:10:09.842462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd27703-27c4-49fe-b3e9-2c268cb7c7ee None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:09.843121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd27703-27c4-49fe-b3e9-2c268cb7c7ee None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:09.843593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd27703-27c4-49fe-b3e9-2c268cb7c7ee None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:09.845281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 14/29] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:09.850510 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb2a41-5277-4f66-81fe-bf586b3097bb None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:09.851043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb2a41-5277-4f66-81fe-bf586b3097bb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:09.852084 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb2a41-5277-4f66-81fe-bf586b3097bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:09.864634 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:09.875330 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b3cb2a41-5277-4f66-81fe-bf586b3097bb None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:09.877296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b3cb2a41-5277-4f66-81fe-bf586b3097bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:09.905595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f527ad4-47b8-4715-a7af-f7dfd33c88af None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:09.916744 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:09.966529 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 15/30] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:09.967796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb2a41-5277-4f66-81fe-bf586b3097bb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:09.976231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6a616-f71d-4924-8925-e2691991b612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:09.977008 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6a616-f71d-4924-8925-e2691991b612 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:09.977285 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6a616-f71d-4924-8925-e2691991b612 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:09.977555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6a616-f71d-4924-8925-e2691991b612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:10.007187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6a616-f71d-4924-8925-e2691991b612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:10.008814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6a616-f71d-4924-8925-e2691991b612 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-e0e6a616-f71d-4924-8925-e2691991b612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:10:10.040531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:10.041474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:10:10.043029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:10:10.043029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:10:10.043029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:10.043029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 16/31] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:10.049220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6a616-f71d-4924-8925-e2691991b612 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:10.166176 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb2a41-5277-4f66-81fe-bf586b3097bb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:10.201436 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6a616-f71d-4924-8925-e2691991b612 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:10.232524 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:10.244482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fd114-9f59-47b8-8116-ce9b3f675d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:10.245441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fd114-9f59-47b8-8116-ce9b3f675d8b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:10.246012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fd114-9f59-47b8-8116-ce9b3f675d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:10.246508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fd114-9f59-47b8-8116-ce9b3f675d8b None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:10.247397 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fd114-9f59-47b8-8116-ce9b3f675d8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:10.247888 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fd114-9f59-47b8-8116-ce9b3f675d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:10.249486 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fd114-9f59-47b8-8116-ce9b3f675d8b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:10.256289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971094f-e780-4b28-a231-cd03fc332e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:10.256936 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971094f-e780-4b28-a231-cd03fc332e03 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:10.257221 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971094f-e780-4b28-a231-cd03fc332e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:10.257498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971094f-e780-4b28-a231-cd03fc332e03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:10.270007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 17/32] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:10.276230 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fd114-9f59-47b8-8116-ce9b3f675d8b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:10.291042 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971094f-e780-4b28-a231-cd03fc332e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7971094f-e780-4b28-a231-cd03fc332e03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:10.297688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:10.298558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:10.299328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:10.299328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:10.299328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:10.299328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:10.299328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:10.299328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:10.299328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:10.299328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:10.299328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:10.299328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:10.299328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:10.299328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:10.299328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 16/33] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:10:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:10.313347 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971094f-e780-4b28-a231-cd03fc332e03 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:10.391652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:10.394546 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fd114-9f59-47b8-8116-ce9b3f675d8b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:10.433748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707e98e-5c12-4deb-9ab1-fa98961f3216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:10.434701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707e98e-5c12-4deb-9ab1-fa98961f3216 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:10.435152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707e98e-5c12-4deb-9ab1-fa98961f3216 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:10.435707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707e98e-5c12-4deb-9ab1-fa98961f3216 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:10.436544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707e98e-5c12-4deb-9ab1-fa98961f3216 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:10.438878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8707e98e-5c12-4deb-9ab1-fa98961f3216 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:10.506591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971094f-e780-4b28-a231-cd03fc332e03 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:10.548773 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56512-beea-49c8-9c28-6d115c2fffe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:10.549589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56512-beea-49c8-9c28-6d115c2fffe5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:10.549960 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56512-beea-49c8-9c28-6d115c2fffe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:10.550354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56512-beea-49c8-9c28-6d115c2fffe5 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:10.551195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed56512-beea-49c8-9c28-6d115c2fffe5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:10.551562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed56512-beea-49c8-9c28-6d115c2fffe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:10.553227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed56512-beea-49c8-9c28-6d115c2fffe5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:10.572480 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 17/34] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:10.573534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56512-beea-49c8-9c28-6d115c2fffe5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:10.604877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 18/35] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:10:10 2022] POST /identity/v3/projects => generated 301 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:10:10.606204 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707e98e-5c12-4deb-9ab1-fa98961f3216 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:10.671573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:10.747054 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707e98e-5c12-4deb-9ab1-fa98961f3216 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:10.766940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:10.784141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56512-beea-49c8-9c28-6d115c2fffe5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:10.824928 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c79175-b8fc-4939-a571-0dfcdc38e2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:10.825717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c79175-b8fc-4939-a571-0dfcdc38e2dc None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:10.826065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c79175-b8fc-4939-a571-0dfcdc38e2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:10.826442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c79175-b8fc-4939-a571-0dfcdc38e2dc None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:10.827211 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c79175-b8fc-4939-a571-0dfcdc38e2dc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:10.828884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c79175-b8fc-4939-a571-0dfcdc38e2dc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:10.846357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 18/36] 10.222.0.55 () {60 vars in 1131 bytes} [Tue Sep 27 17:10:10 2022] POST /identity/v3/roles => generated 225 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:10:10.851776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b319d-102c-4f67-8071-a3b0ad8a0918 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:10.852053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b319d-102c-4f67-8071-a3b0ad8a0918 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:10.852386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b319d-102c-4f67-8071-a3b0ad8a0918 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:10.853964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 19/37] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:10.864621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dee662-86b3-471b-91a7-4e1dfd0fbe39 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:10.864964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dee662-86b3-471b-91a7-4e1dfd0fbe39 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:10.865178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dee662-86b3-471b-91a7-4e1dfd0fbe39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:10.897056 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e1dee662-86b3-471b-91a7-4e1dfd0fbe39 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:10.898966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e1dee662-86b3-471b-91a7-4e1dfd0fbe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:10.904083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b478ec-6a36-401b-b5c6-e3c47aad0e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:10.904847 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b478ec-6a36-401b-b5c6-e3c47aad0e26 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:10.905250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b478ec-6a36-401b-b5c6-e3c47aad0e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:10.905624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b478ec-6a36-401b-b5c6-e3c47aad0e26 None admin] PATH_INFO: `/v3/users/60f027a86a574cfdbd746a543980fbdf` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:10.907100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b478ec-6a36-401b-b5c6-e3c47aad0e26 None admin] RBAC: Authorizing `identity:get_user(user_id=60f027a86a574cfdbd746a543980fbdf)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:10.908953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b478ec-6a36-401b-b5c6-e3c47aad0e26 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:10.911058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 19/38] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:10 2022] GET /identity/v3/users/60f027a86a574cfdbd746a543980fbdf => generated 247 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:10.916947 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b478ec-6a36-401b-b5c6-e3c47aad0e26 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:10.990288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 20/39] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:11.033594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b478ec-6a36-401b-b5c6-e3c47aad0e26 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:11.042632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:11.074352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd067e94-9eea-4ac9-b07c-7b82675d7809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:11.074997 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd067e94-9eea-4ac9-b07c-7b82675d7809 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:11.075275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd067e94-9eea-4ac9-b07c-7b82675d7809 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:11.075543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd067e94-9eea-4ac9-b07c-7b82675d7809 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:11.076774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd067e94-9eea-4ac9-b07c-7b82675d7809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:11.078263 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd067e94-9eea-4ac9-b07c-7b82675d7809 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:11.080091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 20/40] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:11.085462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dee662-86b3-471b-91a7-4e1dfd0fbe39 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:11.088960 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24f158-1bf1-46bd-8dab-19b45ca29847 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:11.089251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24f158-1bf1-46bd-8dab-19b45ca29847 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:11.089533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24f158-1bf1-46bd-8dab-19b45ca29847 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:11.091017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 21/41] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:11.095943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052401c8-c9a7-45be-96fc-3a7b6854a3a8 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:11.096233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052401c8-c9a7-45be-96fc-3a7b6854a3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:11.096507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052401c8-c9a7-45be-96fc-3a7b6854a3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:11.121472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-052401c8-c9a7-45be-96fc-3a7b6854a3a8 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:11.123510 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-052401c8-c9a7-45be-96fc-3a7b6854a3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:11.187054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:11.209324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 22/42] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:11.212413 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dee662-86b3-471b-91a7-4e1dfd0fbe39 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:11.250032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680009e-1cc8-4795-b3eb-149b3f0a6590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:11.250795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680009e-1cc8-4795-b3eb-149b3f0a6590 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:11.251142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680009e-1cc8-4795-b3eb-149b3f0a6590 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:11.251486 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680009e-1cc8-4795-b3eb-149b3f0a6590 None admin] PATH_INFO: `/v3/domains/default/users/60f027a86a574cfdbd746a543980fbdf/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:11.253833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680009e-1cc8-4795-b3eb-149b3f0a6590 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=60f027a86a574cfdbd746a543980fbdf, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:11.255410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680009e-1cc8-4795-b3eb-149b3f0a6590 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:11.268982 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052401c8-c9a7-45be-96fc-3a7b6854a3a8 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:11.282106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 21/43] 10.222.0.55 () {58 vars in 1361 bytes} [Tue Sep 27 17:10:11 2022] PUT /identity/v3/domains/default/users/60f027a86a574cfdbd746a543980fbdf/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:10:11.389272 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052401c8-c9a7-45be-96fc-3a7b6854a3a8 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:11.391859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:11.427643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f66f5-f1b9-42d0-b374-bf2ae5227d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:11.428268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f66f5-f1b9-42d0-b374-bf2ae5227d2f None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:11.428548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f66f5-f1b9-42d0-b374-bf2ae5227d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:11.428821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f66f5-f1b9-42d0-b374-bf2ae5227d2f None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:11.430038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f66f5-f1b9-42d0-b374-bf2ae5227d2f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:11.431510 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f66f5-f1b9-42d0-b374-bf2ae5227d2f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:11.448545 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 23/44] 10.222.0.55 () {60 vars in 1131 bytes} [Tue Sep 27 17:10:11 2022] POST /identity/v3/roles => generated 223 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:10:11.478278 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680009e-1cc8-4795-b3eb-149b3f0a6590 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:11.593299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680009e-1cc8-4795-b3eb-149b3f0a6590 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:11.636924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab32fc0-1a17-4fd7-9d16-a53cfee6ae58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:11.637563 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab32fc0-1a17-4fd7-9d16-a53cfee6ae58 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:11.637842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab32fc0-1a17-4fd7-9d16-a53cfee6ae58 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:11.638143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab32fc0-1a17-4fd7-9d16-a53cfee6ae58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:11.639750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab32fc0-1a17-4fd7-9d16-a53cfee6ae58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:11.641204 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab32fc0-1a17-4fd7-9d16-a53cfee6ae58 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:11.642915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 22/45] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:11.649128 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f66f5-f1b9-42d0-b374-bf2ae5227d2f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:11.752224 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f66f5-f1b9-42d0-b374-bf2ae5227d2f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:11.785552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455b1e0-4e6b-49b7-8f20-90978f8bc0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:11.786189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455b1e0-4e6b-49b7-8f20-90978f8bc0eb None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:11.786470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455b1e0-4e6b-49b7-8f20-90978f8bc0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:11.786742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455b1e0-4e6b-49b7-8f20-90978f8bc0eb None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:11.787387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455b1e0-4e6b-49b7-8f20-90978f8bc0eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:11.789366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455b1e0-4e6b-49b7-8f20-90978f8bc0eb None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:11.814698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 24/46] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:10:11 2022] POST /identity/v3/projects => generated 312 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:10:13.452356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae148e-de5f-4ceb-97fc-231e743bb181 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:13.452893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae148e-de5f-4ceb-97fc-231e743bb181 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:13.452893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ae148e-de5f-4ceb-97fc-231e743bb181 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:13.454312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 23/47] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:13.459651 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6710fff-a651-4f91-8085-32c4d31396dc None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:13.460043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6710fff-a651-4f91-8085-32c4d31396dc None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:13.460208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6710fff-a651-4f91-8085-32c4d31396dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:13.479292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a6710fff-a651-4f91-8085-32c4d31396dc None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:13.480494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a6710fff-a651-4f91-8085-32c4d31396dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:13.492378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:13.548044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 25/48] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:13.826636 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae148e-de5f-4ceb-97fc-231e743bb181 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:13.836906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:13.935802 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ae148e-de5f-4ceb-97fc-231e743bb181 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:13.953277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d500a57-73c2-440b-96bf-f20b06eb19d5 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:13.953572 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d500a57-73c2-440b-96bf-f20b06eb19d5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:13.953856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d500a57-73c2-440b-96bf-f20b06eb19d5 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:13.955248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 26/49] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:10:13.960055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344b3c1-6732-48a7-8be8-4bb58a7a6b44 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:13.960911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344b3c1-6732-48a7-8be8-4bb58a7a6b44 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:13.961250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344b3c1-6732-48a7-8be8-4bb58a7a6b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:13.969982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c3851-951c-440a-a0d3-097b6b8cd9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:13.970632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c3851-951c-440a-a0d3-097b6b8cd9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:13.970912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c3851-951c-440a-a0d3-097b6b8cd9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:13.971184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c3851-951c-440a-a0d3-097b6b8cd9a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:13.979299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b344b3c1-6732-48a7-8be8-4bb58a7a6b44 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:13.980490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b344b3c1-6732-48a7-8be8-4bb58a7a6b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:13.982010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c3851-951c-440a-a0d3-097b6b8cd9a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-9a5c3851-951c-440a-a0d3-097b6b8cd9a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:13.987526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:13.988367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:13.989229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:13.989229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:13.989229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:13.989229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:13.989229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:13.989229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:13.989229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:13.989229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:13.989229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:13.989229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:13.989229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:13.989229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:13.989229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 24/50] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:10:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:13.994246 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c3851-951c-440a-a0d3-097b6b8cd9a8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:14.046865 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 27/51] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:14.059167 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:14.096520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c3851-951c-440a-a0d3-097b6b8cd9a8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:14.130021 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cea02-3c72-4ffc-ad48-88ec9ba97bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:14.130645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cea02-3c72-4ffc-ad48-88ec9ba97bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:14.130924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cea02-3c72-4ffc-ad48-88ec9ba97bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:14.131200 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cea02-3c72-4ffc-ad48-88ec9ba97bb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:14.131863 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cea02-3c72-4ffc-ad48-88ec9ba97bb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:14.132168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cea02-3c72-4ffc-ad48-88ec9ba97bb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:14.134093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cea02-3c72-4ffc-ad48-88ec9ba97bb3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:14.142597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 25/52] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:14.149003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344b3c1-6732-48a7-8be8-4bb58a7a6b44 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:14.224118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:14.246359 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344b3c1-6732-48a7-8be8-4bb58a7a6b44 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:14.279160 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee91dd-9f91-4b43-8631-422ce428024b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:14.279783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee91dd-9f91-4b43-8631-422ce428024b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:14.280063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee91dd-9f91-4b43-8631-422ce428024b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:14.280336 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee91dd-9f91-4b43-8631-422ce428024b None admin] PATH_INFO: `/v3/users/60f027a86a574cfdbd746a543980fbdf` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:14.281453 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee91dd-9f91-4b43-8631-422ce428024b None admin] RBAC: Authorizing `identity:get_user(user_id=60f027a86a574cfdbd746a543980fbdf)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:14.282937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee91dd-9f91-4b43-8631-422ce428024b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:14.284506 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 28/53] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:14 2022] GET /identity/v3/users/60f027a86a574cfdbd746a543980fbdf => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:14.290034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cea02-3c72-4ffc-ad48-88ec9ba97bb3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:14.314705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee91dd-9f91-4b43-8631-422ce428024b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:14.391567 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cea02-3c72-4ffc-ad48-88ec9ba97bb3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:14.413321 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee91dd-9f91-4b43-8631-422ce428024b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:14.416740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:14.424641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb838def-ae6d-47ac-803d-337c5e630ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:14.425266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb838def-ae6d-47ac-803d-337c5e630ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:14.425546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb838def-ae6d-47ac-803d-337c5e630ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:14.425820 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb838def-ae6d-47ac-803d-337c5e630ea3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:14.427191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb838def-ae6d-47ac-803d-337c5e630ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:14.428413 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb838def-ae6d-47ac-803d-337c5e630ea3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:14.430755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 26/54] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:14.436258 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb838def-ae6d-47ac-803d-337c5e630ea3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:14.448156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ac09c-24e7-436e-b832-35435634e749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:14.448792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ac09c-24e7-436e-b832-35435634e749 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:14.449076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ac09c-24e7-436e-b832-35435634e749 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:14.449351 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ac09c-24e7-436e-b832-35435634e749 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:14.450503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ac09c-24e7-436e-b832-35435634e749 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:14.451946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ac09c-24e7-436e-b832-35435634e749 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:14.453523 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 29/55] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:14.458977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ac09c-24e7-436e-b832-35435634e749 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:14.474412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:14.565863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ac09c-24e7-436e-b832-35435634e749 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:14.603112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee7bbd-974c-4d27-80c8-404c4a4bfeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:14.603728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee7bbd-974c-4d27-80c8-404c4a4bfeee None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:14.604012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee7bbd-974c-4d27-80c8-404c4a4bfeee None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:14.604287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee7bbd-974c-4d27-80c8-404c4a4bfeee None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:14.604951 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee7bbd-974c-4d27-80c8-404c4a4bfeee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:14.606448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee7bbd-974c-4d27-80c8-404c4a4bfeee None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:14.634197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 30/56] 10.222.0.55 () {60 vars in 1140 bytes} [Tue Sep 27 17:10:14 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:10:14.647203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb838def-ae6d-47ac-803d-337c5e630ea3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:14.683781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ba39d-9f77-4874-8392-6f9cb8fed8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:14.684406 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ba39d-9f77-4874-8392-6f9cb8fed8fe None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:14.684689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ba39d-9f77-4874-8392-6f9cb8fed8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:14.684963 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ba39d-9f77-4874-8392-6f9cb8fed8fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:14.685714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ba39d-9f77-4874-8392-6f9cb8fed8fe None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.domain.id=default, user.id=60f027a86a574cfdbd746a543980fbdf) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:14.686284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ba39d-9f77-4874-8392-6f9cb8fed8fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:14.687821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ba39d-9f77-4874-8392-6f9cb8fed8fe None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:14.697311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 27/57] 10.222.0.55 () {56 vars in 1322 bytes} [Tue Sep 27 17:10:14 2022] GET /identity/v3/role_assignments?user.id=60f027a86a574cfdbd746a543980fbdf&scope.domain.id=default&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 530 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:16.808993 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77995a4e-9e9d-4ec1-9ed8-d635ce594483 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:16.809507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77995a4e-9e9d-4ec1-9ed8-d635ce594483 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:16.809590 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77995a4e-9e9d-4ec1-9ed8-d635ce594483 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:16.810898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 31/58] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:16.815962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31fdae-94a1-47ff-9a7c-be2c212072ba None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:16.816390 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31fdae-94a1-47ff-9a7c-be2c212072ba None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:16.816511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31fdae-94a1-47ff-9a7c-be2c212072ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:16.834780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9c31fdae-94a1-47ff-9a7c-be2c212072ba None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:16.836482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9c31fdae-94a1-47ff-9a7c-be2c212072ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:16.889671 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:16.901432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 28/59] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:17.167311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77995a4e-9e9d-4ec1-9ed8-d635ce594483 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:17.232226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:17.271358 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77995a4e-9e9d-4ec1-9ed8-d635ce594483 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:17.304411 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19531e4a-7afb-46b7-8760-9f3be4a6ec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:17.305039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19531e4a-7afb-46b7-8760-9f3be4a6ec67 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:17.305312 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19531e4a-7afb-46b7-8760-9f3be4a6ec67 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:17.305579 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19531e4a-7afb-46b7-8760-9f3be4a6ec67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:17.306751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19531e4a-7afb-46b7-8760-9f3be4a6ec67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:17.308198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19531e4a-7afb-46b7-8760-9f3be4a6ec67 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:17.309775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 32/60] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:17.315333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31fdae-94a1-47ff-9a7c-be2c212072ba None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:17.410161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31fdae-94a1-47ff-9a7c-be2c212072ba None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:17.442400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e6f5b-e30f-400d-87c2-51c6e2ce99b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:17.442937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e6f5b-e30f-400d-87c2-51c6e2ce99b6 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:17.443216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e6f5b-e30f-400d-87c2-51c6e2ce99b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:17.443488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e6f5b-e30f-400d-87c2-51c6e2ce99b6 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:17.444131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e6f5b-e30f-400d-87c2-51c6e2ce99b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:17.446213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e6f5b-e30f-400d-87c2-51c6e2ce99b6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:17.472776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 29/61] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:10:17 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:10:20.794475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a9590-1fa7-411d-a5c9-cf55d92ead02 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:20.795504 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a9590-1fa7-411d-a5c9-cf55d92ead02 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:20.795992 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a9590-1fa7-411d-a5c9-cf55d92ead02 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:20.797929 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 33/62] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:20.803332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460660f2-0700-4849-8374-5f0307584019 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:20.804292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460660f2-0700-4849-8374-5f0307584019 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:20.804780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460660f2-0700-4849-8374-5f0307584019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:20.831318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-460660f2-0700-4849-8374-5f0307584019 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:20.832931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-460660f2-0700-4849-8374-5f0307584019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:20.886845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:20.921498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 30/63] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:20.961050 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5507f25-8fd2-421b-aaa5-6b7c2adba9b1 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:20.961666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5507f25-8fd2-421b-aaa5-6b7c2adba9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:20.962195 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5507f25-8fd2-421b-aaa5-6b7c2adba9b1 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:20.964568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 34/64] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:20.970421 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5cf3d-6244-4df2-bc5d-82dc7da4dfca None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:20.970932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5cf3d-6244-4df2-bc5d-82dc7da4dfca None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:20.971436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5cf3d-6244-4df2-bc5d-82dc7da4dfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:20.996467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-06f5cf3d-6244-4df2-bc5d-82dc7da4dfca None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:20.997969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-06f5cf3d-6244-4df2-bc5d-82dc7da4dfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:21.043423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812f35b-51ae-42f2-a89f-a53ab42095fb None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:21.043724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812f35b-51ae-42f2-a89f-a53ab42095fb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:21.043978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812f35b-51ae-42f2-a89f-a53ab42095fb None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:21.045512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 35/65] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:10:21.050983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1d9b5-1aee-4e97-8a80-5448d596af2f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:21.051490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1d9b5-1aee-4e97-8a80-5448d596af2f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:21.051950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1d9b5-1aee-4e97-8a80-5448d596af2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:21.070607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-16e1d9b5-1aee-4e97-8a80-5448d596af2f None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:21.072100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-16e1d9b5-1aee-4e97-8a80-5448d596af2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:21.112615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 31/66] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:21.160402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 36/67] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:21.245843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5cf3d-6244-4df2-bc5d-82dc7da4dfca None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:21.369726 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac27771-fb5a-42e4-a3eb-e1f2d8926f0f None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:21.370443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac27771-fb5a-42e4-a3eb-e1f2d8926f0f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:21.370905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac27771-fb5a-42e4-a3eb-e1f2d8926f0f None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:21.372429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 37/68] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:21.377426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8598870-e403-4ad1-bc50-ce1a4211231d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:21.377711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8598870-e403-4ad1-bc50-ce1a4211231d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:21.377983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8598870-e403-4ad1-bc50-ce1a4211231d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:21.402391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a8598870-e403-4ad1-bc50-ce1a4211231d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:21.403565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a8598870-e403-4ad1-bc50-ce1a4211231d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:21.417487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5cf3d-6244-4df2-bc5d-82dc7da4dfca None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:21.426910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:21.473387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc250b-f37b-4aa0-90d5-81590d91e3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:21.474020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc250b-f37b-4aa0-90d5-81590d91e3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:21.474325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc250b-f37b-4aa0-90d5-81590d91e3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:21.474597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc250b-f37b-4aa0-90d5-81590d91e3f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:21.486746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc250b-f37b-4aa0-90d5-81590d91e3f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-12fc250b-f37b-4aa0-90d5-81590d91e3f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:21.492982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:21.493826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 32/69] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:21.494902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc250b-f37b-4aa0-90d5-81590d91e3f5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:21.497566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:21.515089 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 38/70] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:21.521575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8598870-e403-4ad1-bc50-ce1a4211231d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:21.611828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc250b-f37b-4aa0-90d5-81590d91e3f5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:21.654408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26155e4d-82ef-4340-96c0-3f7b822dbbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:21.654756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8598870-e403-4ad1-bc50-ce1a4211231d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:21.655696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26155e4d-82ef-4340-96c0-3f7b822dbbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:21.656326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26155e4d-82ef-4340-96c0-3f7b822dbbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:21.656786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26155e4d-82ef-4340-96c0-3f7b822dbbf3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:21.670223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26155e4d-82ef-4340-96c0-3f7b822dbbf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-26155e4d-82ef-4340-96c0-3f7b822dbbf3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:21.675348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:21.676304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:21.677072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:21.677072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:21.677072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:21.677072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:21.677072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:21.677072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:21.677072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:21.677072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:21.677072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:21.677072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:21.677072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:10:21.677072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:21.677072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 33/71] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:10:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:21.677705 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26155e4d-82ef-4340-96c0-3f7b822dbbf3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:21.697473 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffeaeda-34f5-4d00-a0e3-9f41f3c5cc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:21.698145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeaeda-34f5-4d00-a0e3-9f41f3c5cc5f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:21.698428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeaeda-34f5-4d00-a0e3-9f41f3c5cc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:21.698698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffeaeda-34f5-4d00-a0e3-9f41f3c5cc5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:21.707899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:21.708952 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffeaeda-34f5-4d00-a0e3-9f41f3c5cc5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-bffeaeda-34f5-4d00-a0e3-9f41f3c5cc5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:21.713871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:21.714898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:21.715864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:21.715864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:21.715864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:21.715864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:21.715864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:21.715864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:21.715864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:21.715864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:21.715864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:21.715864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:21.715864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:21.715864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:21.715864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 39/72] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:10:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:21.716661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18fd84-454d-4923-855d-7d17a8834bb5 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:21.717031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18fd84-454d-4923-855d-7d17a8834bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:21.717126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18fd84-454d-4923-855d-7d17a8834bb5 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:21.718329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 40/73] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:21.722368 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18fd84-454d-4923-855d-7d17a8834bb5 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:21.777768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:21.795446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26155e4d-82ef-4340-96c0-3f7b822dbbf3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:21.828937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18fd84-454d-4923-855d-7d17a8834bb5 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:21.829429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86e93b-69c1-46ee-b7bb-94a191e176c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:21.829984 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86e93b-69c1-46ee-b7bb-94a191e176c2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:21.830297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86e93b-69c1-46ee-b7bb-94a191e176c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:21.830575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86e93b-69c1-46ee-b7bb-94a191e176c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:21.831237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86e93b-69c1-46ee-b7bb-94a191e176c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:21.831539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86e93b-69c1-46ee-b7bb-94a191e176c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:21.832925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86e93b-69c1-46ee-b7bb-94a191e176c2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:21.841701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 34/74] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:21.842333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86e93b-69c1-46ee-b7bb-94a191e176c2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:21.867116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ed833-4b85-4ad8-8b00-f88b5c5b86f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:21.867633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ed833-4b85-4ad8-8b00-f88b5c5b86f2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:21.867915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ed833-4b85-4ad8-8b00-f88b5c5b86f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:21.868188 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ed833-4b85-4ad8-8b00-f88b5c5b86f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:21.868861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ed833-4b85-4ad8-8b00-f88b5c5b86f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:21.869169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ed833-4b85-4ad8-8b00-f88b5c5b86f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:21.870584 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ed833-4b85-4ad8-8b00-f88b5c5b86f2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:21.879167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 41/75] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:10:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:21.879167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffabc5-37f5-438e-8912-7eec01ce8b54 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:21.879167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffabc5-37f5-438e-8912-7eec01ce8b54 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:21.879167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caffabc5-37f5-438e-8912-7eec01ce8b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:21.898395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-caffabc5-37f5-438e-8912-7eec01ce8b54 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:21.899573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-caffabc5-37f5-438e-8912-7eec01ce8b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:21.931131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:21.943429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86e93b-69c1-46ee-b7bb-94a191e176c2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:21.965666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 42/76] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:21.966532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffabc5-37f5-438e-8912-7eec01ce8b54 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:21.976934 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a38eb6-2789-4a0a-82e6-da5c10775a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:21.977854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a38eb6-2789-4a0a-82e6-da5c10775a53 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:21.978218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a38eb6-2789-4a0a-82e6-da5c10775a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:21.978781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a38eb6-2789-4a0a-82e6-da5c10775a53 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:21.979681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a38eb6-2789-4a0a-82e6-da5c10775a53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:21.980051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a38eb6-2789-4a0a-82e6-da5c10775a53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:21.981439 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a38eb6-2789-4a0a-82e6-da5c10775a53 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:21.995385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 35/77] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:21.996166 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a38eb6-2789-4a0a-82e6-da5c10775a53 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:22.015688 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:22.072093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caffabc5-37f5-438e-8912-7eec01ce8b54 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:22.100689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a38eb6-2789-4a0a-82e6-da5c10775a53 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:22.109063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ae4d3-ee06-4e74-b340-e066662a8a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:22.109690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ae4d3-ee06-4e74-b340-e066662a8a1b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:22.109979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ae4d3-ee06-4e74-b340-e066662a8a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:22.110299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ae4d3-ee06-4e74-b340-e066662a8a1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:22.122813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ae4d3-ee06-4e74-b340-e066662a8a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-1f3ae4d3-ee06-4e74-b340-e066662a8a1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:22.127536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:22.128460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:22.129257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:22.129257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:22.129257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:22.129257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:22.129257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:22.129257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:22.129257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:22.129257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:22.129257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:22.129257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:22.129257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:22.129257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:22.129257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 43/78] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:22.130671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ae4d3-ee06-4e74-b340-e066662a8a1b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:22.142233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3894b-1fa6-4201-8ccf-ff204d3e4b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:22.142911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3894b-1fa6-4201-8ccf-ff204d3e4b75 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:22.143141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3894b-1fa6-4201-8ccf-ff204d3e4b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:22.143416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3894b-1fa6-4201-8ccf-ff204d3e4b75 None admin] PATH_INFO: `/v3/users/c8d29301d2f84fcdb72d19727075628d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:22.159465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3894b-1fa6-4201-8ccf-ff204d3e4b75 None admin] RBAC: Authorizing `identity:get_user(user_id=c8d29301d2f84fcdb72d19727075628d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:22.160780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3894b-1fa6-4201-8ccf-ff204d3e4b75 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:22.162603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 36/79] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:21 2022] GET /identity/v3/users/c8d29301d2f84fcdb72d19727075628d => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:22.163315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3894b-1fa6-4201-8ccf-ff204d3e4b75 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:22.173908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:22.261256 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ae4d3-ee06-4e74-b340-e066662a8a1b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:22.263531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:22.292293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3894b-1fa6-4201-8ccf-ff204d3e4b75 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:22.296097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a0d0f-39b0-4984-ab35-07ff57eb10f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:22.296733 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a0d0f-39b0-4984-ab35-07ff57eb10f8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:22.297011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a0d0f-39b0-4984-ab35-07ff57eb10f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:22.297285 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a0d0f-39b0-4984-ab35-07ff57eb10f8 None admin] PATH_INFO: `/v3/users/c8d29301d2f84fcdb72d19727075628d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:22.298479 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a0d0f-39b0-4984-ab35-07ff57eb10f8 None admin] RBAC: Authorizing `identity:get_user(user_id=c8d29301d2f84fcdb72d19727075628d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:22.299946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a0d0f-39b0-4984-ab35-07ff57eb10f8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:22.302280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 44/80] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:22 2022] GET /identity/v3/users/c8d29301d2f84fcdb72d19727075628d => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:22.307483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a0d0f-39b0-4984-ab35-07ff57eb10f8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:22.327515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f640846-f5dd-43e3-a15f-7b0744793407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:22.328141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f640846-f5dd-43e3-a15f-7b0744793407 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:22.328420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f640846-f5dd-43e3-a15f-7b0744793407 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:22.328694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f640846-f5dd-43e3-a15f-7b0744793407 None admin] PATH_INFO: `/v3/users/60f027a86a574cfdbd746a543980fbdf` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:22.329839 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f640846-f5dd-43e3-a15f-7b0744793407 None admin] RBAC: Authorizing `identity:get_user(user_id=60f027a86a574cfdbd746a543980fbdf)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:22.331310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f640846-f5dd-43e3-a15f-7b0744793407 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:22.332970 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 37/81] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:22 2022] GET /identity/v3/users/60f027a86a574cfdbd746a543980fbdf => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:22.333741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f640846-f5dd-43e3-a15f-7b0744793407 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:22.406808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:22.408300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a0d0f-39b0-4984-ab35-07ff57eb10f8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:22.429453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f640846-f5dd-43e3-a15f-7b0744793407 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:22.440632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf4549-5c43-44c3-ae48-e09c3c89b517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:22.441237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf4549-5c43-44c3-ae48-e09c3c89b517 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:22.441520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf4549-5c43-44c3-ae48-e09c3c89b517 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:22.441796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf4549-5c43-44c3-ae48-e09c3c89b517 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:22.442510 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf4549-5c43-44c3-ae48-e09c3c89b517 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:22.442819 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf4549-5c43-44c3-ae48-e09c3c89b517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:22.444727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf4549-5c43-44c3-ae48-e09c3c89b517 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:22.452585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 45/82] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:22.453370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf4549-5c43-44c3-ae48-e09c3c89b517 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:22.463363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ecc82-7d8f-468c-83da-b8e05765ed1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:22.463968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ecc82-7d8f-468c-83da-b8e05765ed1b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:22.464246 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ecc82-7d8f-468c-83da-b8e05765ed1b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:22.464524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590ecc82-7d8f-468c-83da-b8e05765ed1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:22.486717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:22.530887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ecc82-7d8f-468c-83da-b8e05765ed1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:22.532336 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590ecc82-7d8f-468c-83da-b8e05765ed1b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:22.549983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf4549-5c43-44c3-ae48-e09c3c89b517 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:22.582839 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72a5a2-8099-49d8-b369-fae72b6eea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:22.583464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72a5a2-8099-49d8-b369-fae72b6eea38 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:22.583744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72a5a2-8099-49d8-b369-fae72b6eea38 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:22.584020 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72a5a2-8099-49d8-b369-fae72b6eea38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:22.585675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72a5a2-8099-49d8-b369-fae72b6eea38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:22.587151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a72a5a2-8099-49d8-b369-fae72b6eea38 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:22.588715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 46/83] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:22.589476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72a5a2-8099-49d8-b369-fae72b6eea38 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:22.609607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:22.632963 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 38/84] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:22.633787 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ecc82-7d8f-468c-83da-b8e05765ed1b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:22.691646 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a72a5a2-8099-49d8-b369-fae72b6eea38 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:22.693064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:22.726502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b70fe-3f09-4090-a1c7-adf2f104fc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:22.727105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b70fe-3f09-4090-a1c7-adf2f104fc72 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:22.727386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b70fe-3f09-4090-a1c7-adf2f104fc72 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:22.727661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b70fe-3f09-4090-a1c7-adf2f104fc72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:22.728122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590ecc82-7d8f-468c-83da-b8e05765ed1b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:22.761545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595780d-ba83-41e0-9560-83f05fa94b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:22.762168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595780d-ba83-41e0-9560-83f05fa94b77 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:22.762446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595780d-ba83-41e0-9560-83f05fa94b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:22.762718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595780d-ba83-41e0-9560-83f05fa94b77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:22.793596 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b70fe-3f09-4090-a1c7-adf2f104fc72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:22.795104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b70fe-3f09-4090-a1c7-adf2f104fc72 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:22.807335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:22.830878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595780d-ba83-41e0-9560-83f05fa94b77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:22.832345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595780d-ba83-41e0-9560-83f05fa94b77 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:22.893587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 47/85] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:22.894456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b70fe-3f09-4090-a1c7-adf2f104fc72 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:22.896679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:22.928009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 39/86] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:22.929568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595780d-ba83-41e0-9560-83f05fa94b77 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:22.998100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b70fe-3f09-4090-a1c7-adf2f104fc72 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:23.004723 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:23.027343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595780d-ba83-41e0-9560-83f05fa94b77 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:23.030649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0dc22-b9c9-43a8-9d56-3d26bad7aace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:23.031276 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0dc22-b9c9-43a8-9d56-3d26bad7aace None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:23.031555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0dc22-b9c9-43a8-9d56-3d26bad7aace None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:23.031825 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0dc22-b9c9-43a8-9d56-3d26bad7aace None admin] PATH_INFO: `/v3/users/c8d29301d2f84fcdb72d19727075628d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:23.032940 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a0dc22-b9c9-43a8-9d56-3d26bad7aace None admin] RBAC: Authorizing `identity:get_user(user_id=c8d29301d2f84fcdb72d19727075628d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:23.034406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a0dc22-b9c9-43a8-9d56-3d26bad7aace None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:23.035974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 48/87] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:22 2022] GET /identity/v3/users/c8d29301d2f84fcdb72d19727075628d => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:23.037430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0dc22-b9c9-43a8-9d56-3d26bad7aace None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:23.061872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d77c3-3f41-4659-a4a6-e8f59712f03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:23.062503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d77c3-3f41-4659-a4a6-e8f59712f03b None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:23.062780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d77c3-3f41-4659-a4a6-e8f59712f03b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:23.063053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d77c3-3f41-4659-a4a6-e8f59712f03b None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:23.063701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d77c3-3f41-4659-a4a6-e8f59712f03b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:23.065130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d77c3-3f41-4659-a4a6-e8f59712f03b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:23.085500 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 40/88] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:10:22 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:10:23.086182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d77c3-3f41-4659-a4a6-e8f59712f03b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:23.111003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:23.146053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0dc22-b9c9-43a8-9d56-3d26bad7aace None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:23.186523 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77273bb3-c66c-4574-962f-c7a8226458d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:23.187173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77273bb3-c66c-4574-962f-c7a8226458d1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:23.187457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77273bb3-c66c-4574-962f-c7a8226458d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:23.187733 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77273bb3-c66c-4574-962f-c7a8226458d1 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:23.188895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77273bb3-c66c-4574-962f-c7a8226458d1 None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:23.190404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77273bb3-c66c-4574-962f-c7a8226458d1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:23.192099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 49/89] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:23 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:23.192897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77273bb3-c66c-4574-962f-c7a8226458d1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:23.215637 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d77c3-3f41-4659-a4a6-e8f59712f03b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:23.253071 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:23.262135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916df25-19ec-4f46-9e7f-fcc5552c8e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:23.262440 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916df25-19ec-4f46-9e7f-fcc5552c8e8b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:23.262703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916df25-19ec-4f46-9e7f-fcc5552c8e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:23.262975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916df25-19ec-4f46-9e7f-fcc5552c8e8b None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:23.264136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916df25-19ec-4f46-9e7f-fcc5552c8e8b None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:23.266110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916df25-19ec-4f46-9e7f-fcc5552c8e8b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:23.272175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 41/90] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:23 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:23.272175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916df25-19ec-4f46-9e7f-fcc5552c8e8b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:23.317225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77273bb3-c66c-4574-962f-c7a8226458d1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:23.354463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f2284-293b-4e02-9e5f-537feae10830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:23.355132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f2284-293b-4e02-9e5f-537feae10830 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:23.355382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f2284-293b-4e02-9e5f-537feae10830 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:23.355648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f2284-293b-4e02-9e5f-537feae10830 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:23.357627 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f2284-293b-4e02-9e5f-537feae10830 None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:23.359361 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f2284-293b-4e02-9e5f-537feae10830 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:23.361002 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 50/91] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:23 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:23.361860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f2284-293b-4e02-9e5f-537feae10830 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:23.374301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:23.396819 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916df25-19ec-4f46-9e7f-fcc5552c8e8b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:23.436538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60280310-482e-4e4c-a2e7-71f25673f3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:23.437142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60280310-482e-4e4c-a2e7-71f25673f3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:23.437423 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60280310-482e-4e4c-a2e7-71f25673f3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:23.437693 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60280310-482e-4e4c-a2e7-71f25673f3d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:23.488846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f2284-293b-4e02-9e5f-537feae10830 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:23.514385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:23.516925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60280310-482e-4e4c-a2e7-71f25673f3d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:23.519068 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60280310-482e-4e4c-a2e7-71f25673f3d9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:23.532503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a7e8e-b9b5-42d5-996f-9a9059a06d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:23.533167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a7e8e-b9b5-42d5-996f-9a9059a06d9e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:23.533704 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a7e8e-b9b5-42d5-996f-9a9059a06d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:23.533997 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a7e8e-b9b5-42d5-996f-9a9059a06d9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:23.534798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a7e8e-b9b5-42d5-996f-9a9059a06d9e None admin] RBAC: Adding query filter params (role.id=c135afb53eaf43d9a99716160f0ffd25, scope.project.id=00cafd6359074bd5ae371a50effe7297, user.id=c8d29301d2f84fcdb72d19727075628d) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:23.535103 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a7e8e-b9b5-42d5-996f-9a9059a06d9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:23.537267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a7e8e-b9b5-42d5-996f-9a9059a06d9e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:23.545264 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 51/92] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:23 2022] GET /identity/v3/role_assignments?user.id=c8d29301d2f84fcdb72d19727075628d&scope.project.id=00cafd6359074bd5ae371a50effe7297&role.id=c135afb53eaf43d9a99716160f0ffd25 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:23.546859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a7e8e-b9b5-42d5-996f-9a9059a06d9e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:23.702444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:23.735609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 42/93] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:23.736508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60280310-482e-4e4c-a2e7-71f25673f3d9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:23.753193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a7e8e-b9b5-42d5-996f-9a9059a06d9e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:23.801456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88574e0-8b84-4593-ad65-e3e650957e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:23.802043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88574e0-8b84-4593-ad65-e3e650957e52 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:23.802785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88574e0-8b84-4593-ad65-e3e650957e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:23.803372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88574e0-8b84-4593-ad65-e3e650957e52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:23.804216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88574e0-8b84-4593-ad65-e3e650957e52 None admin] RBAC: Adding query filter params (role.id=dfb71501dfed4ce48cf8cb3559cee9d6, scope.project.id=00cafd6359074bd5ae371a50effe7297, user.id=c8d29301d2f84fcdb72d19727075628d) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:23.804563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88574e0-8b84-4593-ad65-e3e650957e52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:23.806746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88574e0-8b84-4593-ad65-e3e650957e52 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:23.814008 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 52/94] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:23 2022] GET /identity/v3/role_assignments?user.id=c8d29301d2f84fcdb72d19727075628d&scope.project.id=00cafd6359074bd5ae371a50effe7297&role.id=dfb71501dfed4ce48cf8cb3559cee9d6 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:23.819609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88574e0-8b84-4593-ad65-e3e650957e52 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:23.880177 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:23.883903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60280310-482e-4e4c-a2e7-71f25673f3d9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:23.955644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353d076-426f-43fa-8ed0-e41bd2f1a071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:23.957425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353d076-426f-43fa-8ed0-e41bd2f1a071 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:23.962822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353d076-426f-43fa-8ed0-e41bd2f1a071 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:23.963517 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353d076-426f-43fa-8ed0-e41bd2f1a071 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:23.967446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353d076-426f-43fa-8ed0-e41bd2f1a071 None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=00cafd6359074bd5ae371a50effe7297, user.id=60f027a86a574cfdbd746a543980fbdf) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:23.968016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353d076-426f-43fa-8ed0-e41bd2f1a071 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:23.969713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353d076-426f-43fa-8ed0-e41bd2f1a071 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:23.990369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 43/95] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:23 2022] GET /identity/v3/role_assignments?user.id=60f027a86a574cfdbd746a543980fbdf&scope.project.id=00cafd6359074bd5ae371a50effe7297&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:24.034134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88574e0-8b84-4593-ad65-e3e650957e52 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:24.085424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b23015-af05-4389-bcc2-bd9a8107cb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:24.085999 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b23015-af05-4389-bcc2-bd9a8107cb1d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:24.086326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b23015-af05-4389-bcc2-bd9a8107cb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:24.086600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b23015-af05-4389-bcc2-bd9a8107cb1d None admin] PATH_INFO: `/v3/projects/dd9619e9978a4e7b8ae0de623e4aebff` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:24.087767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b23015-af05-4389-bcc2-bd9a8107cb1d None admin] RBAC: Authorizing `identity:get_project(project_id=dd9619e9978a4e7b8ae0de623e4aebff)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:24.089240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b23015-af05-4389-bcc2-bd9a8107cb1d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:24.090922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 53/96] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:23 2022] GET /identity/v3/projects/dd9619e9978a4e7b8ae0de623e4aebff => generated 312 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:24.096906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353d076-426f-43fa-8ed0-e41bd2f1a071 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:24.228746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353d076-426f-43fa-8ed0-e41bd2f1a071 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:24.277765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e79031-0bc0-4299-94fc-cdf7e5c4e0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:24.278879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e79031-0bc0-4299-94fc-cdf7e5c4e0aa None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:24.279357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e79031-0bc0-4299-94fc-cdf7e5c4e0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:24.279869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e79031-0bc0-4299-94fc-cdf7e5c4e0aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:24.280847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e79031-0bc0-4299-94fc-cdf7e5c4e0aa None admin] RBAC: Adding query filter params (role.id=c135afb53eaf43d9a99716160f0ffd25, scope.project.id=dd9619e9978a4e7b8ae0de623e4aebff, user.id=c8d29301d2f84fcdb72d19727075628d) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:24.281365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e79031-0bc0-4299-94fc-cdf7e5c4e0aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:24.283102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e79031-0bc0-4299-94fc-cdf7e5c4e0aa None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:24.290400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 44/97] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:24 2022] GET /identity/v3/role_assignments?user.id=c8d29301d2f84fcdb72d19727075628d&scope.project.id=dd9619e9978a4e7b8ae0de623e4aebff&role.id=c135afb53eaf43d9a99716160f0ffd25 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:26.206826 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbe619-8c12-4ccb-94ec-06447e56ca73 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:26.207372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbe619-8c12-4ccb-94ec-06447e56ca73 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:26.207372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbe619-8c12-4ccb-94ec-06447e56ca73 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:26.208697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 54/98] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:26.216293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42def40-a63f-495f-8c76-4e4ba65b2abd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:26.217138 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42def40-a63f-495f-8c76-4e4ba65b2abd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:26.217628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42def40-a63f-495f-8c76-4e4ba65b2abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:26.247070 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f42def40-a63f-495f-8c76-4e4ba65b2abd None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:26.248596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f42def40-a63f-495f-8c76-4e4ba65b2abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:26.273766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:26.277657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804697b5-4536-4649-9847-cebb7835983c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:26.278615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804697b5-4536-4649-9847-cebb7835983c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:26.279122 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804697b5-4536-4649-9847-cebb7835983c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:26.280655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 55/99] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:26.285972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f16522-fe50-4e99-b99c-22708383fb35 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:26.286301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f16522-fe50-4e99-b99c-22708383fb35 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:26.286576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f16522-fe50-4e99-b99c-22708383fb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:26.293521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:26.321434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-32f16522-fe50-4e99-b99c-22708383fb35 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:26.322678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-32f16522-fe50-4e99-b99c-22708383fb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:26.372378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 45/100] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:26.439555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 56/101] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:26.720570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c6306-1cd6-47ed-a121-3d222c31bdf8 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:26.720933 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c6306-1cd6-47ed-a121-3d222c31bdf8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:26.721128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c6306-1cd6-47ed-a121-3d222c31bdf8 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:26.722516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 46/102] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:26.731353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e14e2-40ae-4fcc-9b18-e1069b590c73 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:26.731639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e14e2-40ae-4fcc-9b18-e1069b590c73 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:26.731913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e14e2-40ae-4fcc-9b18-e1069b590c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:26.758493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-db8e14e2-40ae-4fcc-9b18-e1069b590c73 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:26.759675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-db8e14e2-40ae-4fcc-9b18-e1069b590c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:26.796408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c6306-1cd6-47ed-a121-3d222c31bdf8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:26.861119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 57/103] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:26.948228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e14e2-40ae-4fcc-9b18-e1069b590c73 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:26.991800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c6306-1cd6-47ed-a121-3d222c31bdf8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:27.021598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:27.066429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e14e2-40ae-4fcc-9b18-e1069b590c73 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:27.084844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca20c2-e4fd-460d-b6af-26cffe57cebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:27.085488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca20c2-e4fd-460d-b6af-26cffe57cebd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:27.085821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca20c2-e4fd-460d-b6af-26cffe57cebd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:27.086117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca20c2-e4fd-460d-b6af-26cffe57cebd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:27.087283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca20c2-e4fd-460d-b6af-26cffe57cebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:27.088734 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca20c2-e4fd-460d-b6af-26cffe57cebd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:27.091816 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 47/104] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:27.093028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700404d7-46c4-4bd2-8bd8-143d09c5478f None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:27.093308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700404d7-46c4-4bd2-8bd8-143d09c5478f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:27.093580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700404d7-46c4-4bd2-8bd8-143d09c5478f None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:27.095823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 48/105] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:27.104505 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700404d7-46c4-4bd2-8bd8-143d09c5478f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:27.130229 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b140ed-616b-4258-8b8e-f768a0787042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:27.130877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b140ed-616b-4258-8b8e-f768a0787042 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:27.131723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b140ed-616b-4258-8b8e-f768a0787042 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:27.132002 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b140ed-616b-4258-8b8e-f768a0787042 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:27.150259 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:27.150422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b140ed-616b-4258-8b8e-f768a0787042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-d4b140ed-616b-4258-8b8e-f768a0787042 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:27.156541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:27.157390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:27.158182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:27.158182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:27.158182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:27.158182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:27.158182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:27.158182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:27.158182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:27.158182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:27.158182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:27.158182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:27.158182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:27.158182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:27.160109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 58/106] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:27.164240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bc48b-f8d6-433b-9e56-567e0cfc4a69 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:27.164520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bc48b-f8d6-433b-9e56-567e0cfc4a69 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:27.164793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895bc48b-f8d6-433b-9e56-567e0cfc4a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:27.202560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-895bc48b-f8d6-433b-9e56-567e0cfc4a69 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:27.203823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-895bc48b-f8d6-433b-9e56-567e0cfc4a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:27.282268 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700404d7-46c4-4bd2-8bd8-143d09c5478f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:27.297058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 59/107] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:27.297990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895bc48b-f8d6-433b-9e56-567e0cfc4a69 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:27.349191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff7b37-20e4-4c4e-b123-e473565066d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:27.349805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff7b37-20e4-4c4e-b123-e473565066d8 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:27.350683 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff7b37-20e4-4c4e-b123-e473565066d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:27.350957 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff7b37-20e4-4c4e-b123-e473565066d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:27.351615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff7b37-20e4-4c4e-b123-e473565066d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:27.353059 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff7b37-20e4-4c4e-b123-e473565066d8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:27.388078 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:27.399019 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 49/108] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:10:27 2022] POST /identity/v3/projects => generated 302 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:10:27.400117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff7b37-20e4-4c4e-b123-e473565066d8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:27.454899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895bc48b-f8d6-433b-9e56-567e0cfc4a69 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:27.507103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:27.529703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453152b4-4031-41ba-8bcf-5039e425cba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:27.530419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453152b4-4031-41ba-8bcf-5039e425cba7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:27.530757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453152b4-4031-41ba-8bcf-5039e425cba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:27.530979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453152b4-4031-41ba-8bcf-5039e425cba7 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:27.531750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453152b4-4031-41ba-8bcf-5039e425cba7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:27.532063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453152b4-4031-41ba-8bcf-5039e425cba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:27.533683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453152b4-4031-41ba-8bcf-5039e425cba7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:27.538034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff7b37-20e4-4c4e-b123-e473565066d8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:27.555779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 60/109] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:27.555779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef4ee5-f56e-4757-bc77-5ab9915f03db None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:27.555779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef4ee5-f56e-4757-bc77-5ab9915f03db None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:27.555779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef4ee5-f56e-4757-bc77-5ab9915f03db None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:27.555779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 61/110] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:27.570218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef4ee5-f56e-4757-bc77-5ab9915f03db None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:27.596790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b12c8b-0b0b-4de8-b99b-7ce3c9196c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:27.599027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b12c8b-0b0b-4de8-b99b-7ce3c9196c4e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:27.599681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b12c8b-0b0b-4de8-b99b-7ce3c9196c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:27.602391 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b12c8b-0b0b-4de8-b99b-7ce3c9196c4e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:27.632858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b12c8b-0b0b-4de8-b99b-7ce3c9196c4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d7b12c8b-0b0b-4de8-b99b-7ce3c9196c4e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:27.640298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:27.641865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 50/111] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:27.643230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f93c6f-118e-4260-895a-caa5a7a4305c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:27.645579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f93c6f-118e-4260-895a-caa5a7a4305c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:27.645579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f93c6f-118e-4260-895a-caa5a7a4305c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:27.672412 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e8f93c6f-118e-4260-895a-caa5a7a4305c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:27.673607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e8f93c6f-118e-4260-895a-caa5a7a4305c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:27.759851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef4ee5-f56e-4757-bc77-5ab9915f03db None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:27.761705 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 51/112] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:27.767023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f93c6f-118e-4260-895a-caa5a7a4305c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:27.776099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:27.800274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb0eba-b990-41ae-ad60-ad8b453083f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:27.800832 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb0eba-b990-41ae-ad60-ad8b453083f6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:27.801114 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb0eba-b990-41ae-ad60-ad8b453083f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:27.801389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb0eba-b990-41ae-ad60-ad8b453083f6 None admin] PATH_INFO: `/v3/users/c8d29301d2f84fcdb72d19727075628d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:27.802677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb0eba-b990-41ae-ad60-ad8b453083f6 None admin] RBAC: Authorizing `identity:get_user(user_id=c8d29301d2f84fcdb72d19727075628d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:27.805068 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb0eba-b990-41ae-ad60-ad8b453083f6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:27.810793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 62/113] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:27 2022] GET /identity/v3/users/c8d29301d2f84fcdb72d19727075628d => generated 275 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:27.811663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb0eba-b990-41ae-ad60-ad8b453083f6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:27.852644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:27.891557 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f93c6f-118e-4260-895a-caa5a7a4305c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:27.932117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549d060-e88f-4bc3-a0f8-b6dc1bf4b5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:27.932811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549d060-e88f-4bc3-a0f8-b6dc1bf4b5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:27.933011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549d060-e88f-4bc3-a0f8-b6dc1bf4b5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:27.933356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549d060-e88f-4bc3-a0f8-b6dc1bf4b5d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:27.934043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549d060-e88f-4bc3-a0f8-b6dc1bf4b5d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:27.934386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549d060-e88f-4bc3-a0f8-b6dc1bf4b5d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:27.935892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549d060-e88f-4bc3-a0f8-b6dc1bf4b5d6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:27.941497 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb0eba-b990-41ae-ad60-ad8b453083f6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:27.950158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 52/114] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:27.951538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212732d-9e83-483c-b927-75c11626ef85 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:27.951904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212732d-9e83-483c-b927-75c11626ef85 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:27.952565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212732d-9e83-483c-b927-75c11626ef85 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:27.955190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 53/115] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:27.956035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212732d-9e83-483c-b927-75c11626ef85 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:27.984737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9034989-8362-45fd-9375-65794fa66570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:27.985900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9034989-8362-45fd-9375-65794fa66570 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:27.986406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9034989-8362-45fd-9375-65794fa66570 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:27.986835 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9034989-8362-45fd-9375-65794fa66570 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:28.000915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9034989-8362-45fd-9375-65794fa66570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-f9034989-8362-45fd-9375-65794fa66570 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:28.008010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:28.008860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:28.009619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:28.009619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:28.009619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:28.009619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:28.009619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:28.009619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:28.009619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:28.009619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:28.009619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:28.009619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:28.009619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:28.009619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:28.009619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 63/116] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:10:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:28.010364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9034989-8362-45fd-9375-65794fa66570 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:28.073850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:28.095540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212732d-9e83-483c-b927-75c11626ef85 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:28.130856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:28.137285 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9034989-8362-45fd-9375-65794fa66570 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:28.138364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2243f5-b405-42ba-8f48-e65a441fe859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:28.139331 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2243f5-b405-42ba-8f48-e65a441fe859 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:28.140426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2243f5-b405-42ba-8f48-e65a441fe859 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:28.140712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2243f5-b405-42ba-8f48-e65a441fe859 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:28.141886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2243f5-b405-42ba-8f48-e65a441fe859 None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:28.143399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2243f5-b405-42ba-8f48-e65a441fe859 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:28.145695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 54/117] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:27 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:28.147154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdbb4b-428b-4f24-8308-4280244f5cb5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:28.147454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdbb4b-428b-4f24-8308-4280244f5cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:28.147725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdbb4b-428b-4f24-8308-4280244f5cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:28.169273 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a1bdbb4b-428b-4f24-8308-4280244f5cb5 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:28.171075 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a1bdbb4b-428b-4f24-8308-4280244f5cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:28.178186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef182d37-0819-4d59-93a6-66ec6667274f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:28.178883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef182d37-0819-4d59-93a6-66ec6667274f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:28.179170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef182d37-0819-4d59-93a6-66ec6667274f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:28.179449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef182d37-0819-4d59-93a6-66ec6667274f None admin] PATH_INFO: `/v3/users/2e0d857d7ed04548a601eba89bcc3d8e` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:28.194925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef182d37-0819-4d59-93a6-66ec6667274f None admin] RBAC: Authorizing `identity:get_user(user_id=2e0d857d7ed04548a601eba89bcc3d8e)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:28.197333 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef182d37-0819-4d59-93a6-66ec6667274f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:28.199394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 64/118] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:28 2022] GET /identity/v3/users/2e0d857d7ed04548a601eba89bcc3d8e => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:28.200544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef182d37-0819-4d59-93a6-66ec6667274f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:28.254475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 55/119] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:28.255595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdbb4b-428b-4f24-8308-4280244f5cb5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:28.306225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef182d37-0819-4d59-93a6-66ec6667274f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:28.316213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:28.343942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ecfd3-ce90-4de2-a467-92699de063b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:28.344928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ecfd3-ce90-4de2-a467-92699de063b1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:28.345532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ecfd3-ce90-4de2-a467-92699de063b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:28.345980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ecfd3-ce90-4de2-a467-92699de063b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:28.346855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ecfd3-ce90-4de2-a467-92699de063b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:28.347335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ecfd3-ce90-4de2-a467-92699de063b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:28.349007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ecfd3-ce90-4de2-a467-92699de063b1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:28.357755 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 65/120] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:28.359154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ecfd3-ce90-4de2-a467-92699de063b1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:28.390242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdbb4b-428b-4f24-8308-4280244f5cb5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:28.423390 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:28.433349 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee147d1-0f17-4db9-8c7c-8ea8177e04e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:28.434017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee147d1-0f17-4db9-8c7c-8ea8177e04e1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:28.434337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee147d1-0f17-4db9-8c7c-8ea8177e04e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:28.434614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee147d1-0f17-4db9-8c7c-8ea8177e04e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:28.448429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee147d1-0f17-4db9-8c7c-8ea8177e04e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7ee147d1-0f17-4db9-8c7c-8ea8177e04e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:28.455089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:28.456039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:28.456833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:28.456833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:28.456833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:28.456833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:28.456833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:28.456833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:28.456833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:28.456833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:28.456833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:28.456833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:28.456833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:28.456833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:28.456833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 56/121] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:28.457473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee147d1-0f17-4db9-8c7c-8ea8177e04e1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:28.486331 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ecfd3-ce90-4de2-a467-92699de063b1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:28.533734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86b57f-1f8c-4fcd-8b91-cd5f96f9ca79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:28.534491 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86b57f-1f8c-4fcd-8b91-cd5f96f9ca79 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:28.534776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86b57f-1f8c-4fcd-8b91-cd5f96f9ca79 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:28.535059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86b57f-1f8c-4fcd-8b91-cd5f96f9ca79 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297/users/c8d29301d2f84fcdb72d19727075628d/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:28.538014 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86b57f-1f8c-4fcd-8b91-cd5f96f9ca79 None admin] RBAC: Authorizing `identity:create_grant(project_id=00cafd6359074bd5ae371a50effe7297, user_id=c8d29301d2f84fcdb72d19727075628d, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:28.540802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86b57f-1f8c-4fcd-8b91-cd5f96f9ca79 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:28.564099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 66/122] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:10:28 2022] PUT /identity/v3/projects/00cafd6359074bd5ae371a50effe7297/users/c8d29301d2f84fcdb72d19727075628d/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:10:28.565303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86b57f-1f8c-4fcd-8b91-cd5f96f9ca79 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:28.580591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee147d1-0f17-4db9-8c7c-8ea8177e04e1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:28.603880 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:28.624955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c94ed6-3dd6-4d3d-ac5e-5a98e3b5c674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:28.626165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c94ed6-3dd6-4d3d-ac5e-5a98e3b5c674 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:28.626722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c94ed6-3dd6-4d3d-ac5e-5a98e3b5c674 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:28.627282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c94ed6-3dd6-4d3d-ac5e-5a98e3b5c674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:28.704669 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:28.710877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86b57f-1f8c-4fcd-8b91-cd5f96f9ca79 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:28.721162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c94ed6-3dd6-4d3d-ac5e-5a98e3b5c674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:28.723038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c94ed6-3dd6-4d3d-ac5e-5a98e3b5c674 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:28.763889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4cac3-8ed8-40f2-b4bd-a978eac8d9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:28.764772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4cac3-8ed8-40f2-b4bd-a978eac8d9ae None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:28.765186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4cac3-8ed8-40f2-b4bd-a978eac8d9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:28.765625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4cac3-8ed8-40f2-b4bd-a978eac8d9ae None admin] PATH_INFO: `/v3/users/60f027a86a574cfdbd746a543980fbdf` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:28.766961 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4cac3-8ed8-40f2-b4bd-a978eac8d9ae None admin] RBAC: Authorizing `identity:get_user(user_id=60f027a86a574cfdbd746a543980fbdf)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:28.768761 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4cac3-8ed8-40f2-b4bd-a978eac8d9ae None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:28.770197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 67/123] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:28 2022] GET /identity/v3/users/60f027a86a574cfdbd746a543980fbdf => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:28.771003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4cac3-8ed8-40f2-b4bd-a978eac8d9ae None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:28.840297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 57/124] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:28.842939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c94ed6-3dd6-4d3d-ac5e-5a98e3b5c674 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:28.886627 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:28.899040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4cac3-8ed8-40f2-b4bd-a978eac8d9ae None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:28.936931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e2751-ec9a-4957-a6aa-1d033dca14f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:28.937543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e2751-ec9a-4957-a6aa-1d033dca14f2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:28.937822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e2751-ec9a-4957-a6aa-1d033dca14f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:28.938111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e2751-ec9a-4957-a6aa-1d033dca14f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:28.938785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e2751-ec9a-4957-a6aa-1d033dca14f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:28.939085 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e2751-ec9a-4957-a6aa-1d033dca14f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:28.939415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:28.941560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e2751-ec9a-4957-a6aa-1d033dca14f2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:28.951762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 68/125] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:28.952293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c94ed6-3dd6-4d3d-ac5e-5a98e3b5c674 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:28.953101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e2751-ec9a-4957-a6aa-1d033dca14f2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:28.993433 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4187a-ad56-483f-935a-d4a3eeccbdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:28.994085 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4187a-ad56-483f-935a-d4a3eeccbdf9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:28.994368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4187a-ad56-483f-935a-d4a3eeccbdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:28.994642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4187a-ad56-483f-935a-d4a3eeccbdf9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:29.006612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4187a-ad56-483f-935a-d4a3eeccbdf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-31b4187a-ad56-483f-935a-d4a3eeccbdf9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:29.012513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:29.013572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 58/126] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:10:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:29.014542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4187a-ad56-483f-935a-d4a3eeccbdf9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:29.064018 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e2751-ec9a-4957-a6aa-1d033dca14f2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:29.102620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4ed20-226e-42c0-ab3c-0961cc3a240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:29.103631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4ed20-226e-42c0-ab3c-0961cc3a240c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:29.104105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4ed20-226e-42c0-ab3c-0961cc3a240c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:29.104545 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4ed20-226e-42c0-ab3c-0961cc3a240c None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:29.105925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4ed20-226e-42c0-ab3c-0961cc3a240c None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:29.107698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4ed20-226e-42c0-ab3c-0961cc3a240c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:29.109919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 69/127] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:28 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:29.111086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4ed20-226e-42c0-ab3c-0961cc3a240c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:29.121684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4187a-ad56-483f-935a-d4a3eeccbdf9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:29.136032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:29.163425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746f21b-c356-408d-ab72-f7271d44eba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:29.164321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746f21b-c356-408d-ab72-f7271d44eba1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:29.164600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746f21b-c356-408d-ab72-f7271d44eba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:29.164871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746f21b-c356-408d-ab72-f7271d44eba1 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:29.166217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746f21b-c356-408d-ab72-f7271d44eba1 None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:29.167716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746f21b-c356-408d-ab72-f7271d44eba1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:29.174030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 59/128] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:29 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:29.174030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746f21b-c356-408d-ab72-f7271d44eba1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:29.190924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:29.236582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4ed20-226e-42c0-ab3c-0961cc3a240c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:29.276584 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0f79b-3165-476f-a0d2-37d700027aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:29.277499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0f79b-3165-476f-a0d2-37d700027aaa None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:29.277948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0f79b-3165-476f-a0d2-37d700027aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:29.278470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0f79b-3165-476f-a0d2-37d700027aaa None admin] PATH_INFO: `/v3/users/c8d29301d2f84fcdb72d19727075628d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:29.279608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0f79b-3165-476f-a0d2-37d700027aaa None admin] RBAC: Authorizing `identity:get_user(user_id=c8d29301d2f84fcdb72d19727075628d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:29.281052 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0f79b-3165-476f-a0d2-37d700027aaa None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:29.285216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 70/129] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:29 2022] GET /identity/v3/users/c8d29301d2f84fcdb72d19727075628d => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:29.285216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0f79b-3165-476f-a0d2-37d700027aaa None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:29.298265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746f21b-c356-408d-ab72-f7271d44eba1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:29.343229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32605627-b8c0-40ac-8be7-4e8f3d203714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:29.344131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32605627-b8c0-40ac-8be7-4e8f3d203714 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:29.344548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32605627-b8c0-40ac-8be7-4e8f3d203714 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:29.344931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32605627-b8c0-40ac-8be7-4e8f3d203714 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:29.345705 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32605627-b8c0-40ac-8be7-4e8f3d203714 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:29.346142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32605627-b8c0-40ac-8be7-4e8f3d203714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:29.347633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32605627-b8c0-40ac-8be7-4e8f3d203714 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:29.356693 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 60/130] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:10:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:29.357478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32605627-b8c0-40ac-8be7-4e8f3d203714 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:29.396509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:29.411342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0f79b-3165-476f-a0d2-37d700027aaa None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:29.454240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea8db5-23cb-4b41-8686-94694fb919eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:29.455211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea8db5-23cb-4b41-8686-94694fb919eb None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:29.456085 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea8db5-23cb-4b41-8686-94694fb919eb None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:29.456475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea8db5-23cb-4b41-8686-94694fb919eb None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297/users/60f027a86a574cfdbd746a543980fbdf/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:29.457984 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:29.460277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea8db5-23cb-4b41-8686-94694fb919eb None admin] RBAC: Authorizing `identity:create_grant(project_id=00cafd6359074bd5ae371a50effe7297, user_id=60f027a86a574cfdbd746a543980fbdf, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:29.462390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea8db5-23cb-4b41-8686-94694fb919eb None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:29.481717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 71/131] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:10:29 2022] PUT /identity/v3/projects/00cafd6359074bd5ae371a50effe7297/users/60f027a86a574cfdbd746a543980fbdf/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:10:29.482586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea8db5-23cb-4b41-8686-94694fb919eb None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:29.492642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32605627-b8c0-40ac-8be7-4e8f3d203714 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:29.532135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac42f4-f861-45a1-9549-fe5f08edf3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:29.533018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac42f4-f861-45a1-9549-fe5f08edf3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:29.533466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac42f4-f861-45a1-9549-fe5f08edf3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:29.533954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac42f4-f861-45a1-9549-fe5f08edf3a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:29.534877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac42f4-f861-45a1-9549-fe5f08edf3a6 None admin] RBAC: Adding query filter params (role.id=a002aab559bb446593ca855874db44a2, scope.project.id=00cafd6359074bd5ae371a50effe7297, user.id=2e0d857d7ed04548a601eba89bcc3d8e) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:29.535381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac42f4-f861-45a1-9549-fe5f08edf3a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:29.537144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac42f4-f861-45a1-9549-fe5f08edf3a6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:29.548246 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 61/132] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:29 2022] GET /identity/v3/role_assignments?user.id=2e0d857d7ed04548a601eba89bcc3d8e&scope.project.id=00cafd6359074bd5ae371a50effe7297&role.id=a002aab559bb446593ca855874db44a2 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:29.549666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac42f4-f861-45a1-9549-fe5f08edf3a6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:29.624791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea8db5-23cb-4b41-8686-94694fb919eb None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:29.670250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab4387-3f99-4e42-99dc-a543a1225a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:29.670250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab4387-3f99-4e42-99dc-a543a1225a4f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:29.670250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab4387-3f99-4e42-99dc-a543a1225a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:29.670620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab4387-3f99-4e42-99dc-a543a1225a4f None admin] PATH_INFO: `/v3/projects/dd9619e9978a4e7b8ae0de623e4aebff` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:29.671710 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab4387-3f99-4e42-99dc-a543a1225a4f None admin] RBAC: Authorizing `identity:get_project(project_id=dd9619e9978a4e7b8ae0de623e4aebff)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:29.674286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab4387-3f99-4e42-99dc-a543a1225a4f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:29.676694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 72/133] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:29 2022] GET /identity/v3/projects/dd9619e9978a4e7b8ae0de623e4aebff => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:29.683616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab4387-3f99-4e42-99dc-a543a1225a4f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:29.715877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:29.769706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac42f4-f861-45a1-9549-fe5f08edf3a6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:29.849068 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0c0cc-4f9f-4aa4-8beb-95ac85a886ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:29.850409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0c0cc-4f9f-4aa4-8beb-95ac85a886ce None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:29.850694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0c0cc-4f9f-4aa4-8beb-95ac85a886ce None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:29.850985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0c0cc-4f9f-4aa4-8beb-95ac85a886ce None admin] PATH_INFO: `/v3/users/c8d29301d2f84fcdb72d19727075628d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:29.852162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0c0cc-4f9f-4aa4-8beb-95ac85a886ce None admin] RBAC: Authorizing `identity:get_user(user_id=c8d29301d2f84fcdb72d19727075628d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:29.853636 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0c0cc-4f9f-4aa4-8beb-95ac85a886ce None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:29.855634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 62/134] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:29 2022] GET /identity/v3/users/c8d29301d2f84fcdb72d19727075628d => generated 275 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:29.856289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab4387-3f99-4e42-99dc-a543a1225a4f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:29.860104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119ea0e-3d09-4b41-8812-baca316fdddd None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:29.860395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119ea0e-3d09-4b41-8812-baca316fdddd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:29.860675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a119ea0e-3d09-4b41-8812-baca316fdddd None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 63/135] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:29.865769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119ea0e-3d09-4b41-8812-baca316fdddd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:29.883294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:29.903544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed018a04-9fc4-40b6-ae0b-b9198f6cbc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:29.904454 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed018a04-9fc4-40b6-ae0b-b9198f6cbc3c None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:29.905555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed018a04-9fc4-40b6-ae0b-b9198f6cbc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:29.906156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed018a04-9fc4-40b6-ae0b-b9198f6cbc3c None admin] PATH_INFO: `/v3/projects/dd9619e9978a4e7b8ae0de623e4aebff/users/c8d29301d2f84fcdb72d19727075628d/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:29.910600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed018a04-9fc4-40b6-ae0b-b9198f6cbc3c None admin] RBAC: Authorizing `identity:create_grant(project_id=dd9619e9978a4e7b8ae0de623e4aebff, user_id=c8d29301d2f84fcdb72d19727075628d, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:29.912894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed018a04-9fc4-40b6-ae0b-b9198f6cbc3c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:29.931428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 73/136] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:10:29 2022] PUT /identity/v3/projects/dd9619e9978a4e7b8ae0de623e4aebff/users/c8d29301d2f84fcdb72d19727075628d/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:10:29.933021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da291ad3-9ece-435c-afd3-5fd88fa9dcc7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:29.933445 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da291ad3-9ece-435c-afd3-5fd88fa9dcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:29.933975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da291ad3-9ece-435c-afd3-5fd88fa9dcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:29.955505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-da291ad3-9ece-435c-afd3-5fd88fa9dcc7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:29.957306 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-da291ad3-9ece-435c-afd3-5fd88fa9dcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:29.996570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a119ea0e-3d09-4b41-8812-baca316fdddd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:30.033578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d00660-39be-446f-b0e3-8d00003894b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:30.034326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d00660-39be-446f-b0e3-8d00003894b5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:30.034608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d00660-39be-446f-b0e3-8d00003894b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:30.034880 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d00660-39be-446f-b0e3-8d00003894b5 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:30.036078 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d00660-39be-446f-b0e3-8d00003894b5 None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:30.038310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d00660-39be-446f-b0e3-8d00003894b5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:30.039518 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 74/137] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:30.045555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 64/138] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:29 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:30.051251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da291ad3-9ece-435c-afd3-5fd88fa9dcc7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:30.074478 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:30.188244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da291ad3-9ece-435c-afd3-5fd88fa9dcc7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:30.230198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66753f1f-5036-4151-8cdd-22cae581d5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:30.230871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66753f1f-5036-4151-8cdd-22cae581d5f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:30.231150 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66753f1f-5036-4151-8cdd-22cae581d5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:30.231425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66753f1f-5036-4151-8cdd-22cae581d5f1 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297/users/c8d29301d2f84fcdb72d19727075628d/roles/dfb71501dfed4ce48cf8cb3559cee9d6` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:30.233895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66753f1f-5036-4151-8cdd-22cae581d5f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=00cafd6359074bd5ae371a50effe7297, user_id=c8d29301d2f84fcdb72d19727075628d, role_id=dfb71501dfed4ce48cf8cb3559cee9d6)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:30.235570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66753f1f-5036-4151-8cdd-22cae581d5f1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:30.260804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 75/139] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:10:30 2022] PUT /identity/v3/projects/00cafd6359074bd5ae371a50effe7297/users/c8d29301d2f84fcdb72d19727075628d/roles/dfb71501dfed4ce48cf8cb3559cee9d6 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:10:30.545412 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d00660-39be-446f-b0e3-8d00003894b5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:30.647657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:30.670336 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d00660-39be-446f-b0e3-8d00003894b5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:30.707540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfc7ca-9477-4199-83a1-eab10ce893b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:30.708177 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfc7ca-9477-4199-83a1-eab10ce893b4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:30.708463 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfc7ca-9477-4199-83a1-eab10ce893b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:30.708742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfc7ca-9477-4199-83a1-eab10ce893b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:30.709956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbfc7ca-9477-4199-83a1-eab10ce893b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:30.711492 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbfc7ca-9477-4199-83a1-eab10ce893b4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:30.713080 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 65/140] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:30.718698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66753f1f-5036-4151-8cdd-22cae581d5f1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:30.836567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:30.876324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66753f1f-5036-4151-8cdd-22cae581d5f1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:30.911257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48a714-8be2-453e-afdb-cdca127e1df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:30.911982 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48a714-8be2-453e-afdb-cdca127e1df1 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:30.912301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48a714-8be2-453e-afdb-cdca127e1df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:30.912575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48a714-8be2-453e-afdb-cdca127e1df1 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:30.913232 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48a714-8be2-453e-afdb-cdca127e1df1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:30.914727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48a714-8be2-453e-afdb-cdca127e1df1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:30.944244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 76/141] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:10:30 2022] POST /identity/v3/users => generated 283 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:10:31.611450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998427fe-a3d1-4527-8fd6-2b707492a974 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:31.611946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998427fe-a3d1-4527-8fd6-2b707492a974 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:31.612043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998427fe-a3d1-4527-8fd6-2b707492a974 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:31.613465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 66/142] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:31.618763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdfc21-ff7c-4665-a245-1e07b749ed4d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:31.619045 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdfc21-ff7c-4665-a245-1e07b749ed4d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:31.619312 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cdfc21-ff7c-4665-a245-1e07b749ed4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:31.643390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-76cdfc21-ff7c-4665-a245-1e07b749ed4d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:31.644574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-76cdfc21-ff7c-4665-a245-1e07b749ed4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:31.731690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 77/143] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:32.275036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998427fe-a3d1-4527-8fd6-2b707492a974 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:32.371912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132878dc-01fc-467d-9f34-fa157b3147fe None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:32.372190 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132878dc-01fc-467d-9f34-fa157b3147fe None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:32.372460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132878dc-01fc-467d-9f34-fa157b3147fe None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:32.373941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 78/144] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:10:32.379502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f1a77-4b57-4a6d-9b30-bf30453b85f7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:32.379776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f1a77-4b57-4a6d-9b30-bf30453b85f7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:32.380045 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f1a77-4b57-4a6d-9b30-bf30453b85f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:32.408604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998427fe-a3d1-4527-8fd6-2b707492a974 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:32.422351 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d38f1a77-4b57-4a6d-9b30-bf30453b85f7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:32.423945 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d38f1a77-4b57-4a6d-9b30-bf30453b85f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:32.479025 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a671c-3912-453f-b43b-7b0aca778c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:32.480572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a671c-3912-453f-b43b-7b0aca778c88 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:32.480867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a671c-3912-453f-b43b-7b0aca778c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:32.481144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a671c-3912-453f-b43b-7b0aca778c88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:32.507361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a671c-3912-453f-b43b-7b0aca778c88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-da8a671c-3912-453f-b43b-7b0aca778c88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:32.519603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:32.520613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:32.521965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:32.521965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:32.521965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:32.521965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:32.521965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:32.521965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:32.521965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:32.521965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:32.521965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:32.521965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:32.521965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:32.521965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:32.521965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 67/145] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:32.522724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:32.528335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a671c-3912-453f-b43b-7b0aca778c88 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:32.546457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:32.548142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 79/146] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:32.691293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a671c-3912-453f-b43b-7b0aca778c88 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:32.733464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a14173-b009-49c9-a11b-140b6accf987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:32.734095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a14173-b009-49c9-a11b-140b6accf987 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:32.734380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a14173-b009-49c9-a11b-140b6accf987 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:32.734653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a14173-b009-49c9-a11b-140b6accf987 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:32.735333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a14173-b009-49c9-a11b-140b6accf987 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:32.735633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a14173-b009-49c9-a11b-140b6accf987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:32.737016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a14173-b009-49c9-a11b-140b6accf987 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:32.746660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 68/147] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:32.754951 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f1a77-4b57-4a6d-9b30-bf30453b85f7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:32.895633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f1a77-4b57-4a6d-9b30-bf30453b85f7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:32.899544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c3306-49ae-42a9-bce2-7073bad69308 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:32.899827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c3306-49ae-42a9-bce2-7073bad69308 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:32.900101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c3306-49ae-42a9-bce2-7073bad69308 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:32.901710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 69/148] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:10:32.907312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6731e-0fb3-4fd5-8ea0-b3696cfd582d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:32.907593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6731e-0fb3-4fd5-8ea0-b3696cfd582d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:32.907865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6731e-0fb3-4fd5-8ea0-b3696cfd582d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:32.939174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-50e6731e-0fb3-4fd5-8ea0-b3696cfd582d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:32.940394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-50e6731e-0fb3-4fd5-8ea0-b3696cfd582d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:32.942896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460edae-89bc-455c-aa1d-ce95ba270a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:32.944170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460edae-89bc-455c-aa1d-ce95ba270a8f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:32.944456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460edae-89bc-455c-aa1d-ce95ba270a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:32.944724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460edae-89bc-455c-aa1d-ce95ba270a8f None admin] PATH_INFO: `/v3/users/c8d29301d2f84fcdb72d19727075628d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:32.945874 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460edae-89bc-455c-aa1d-ce95ba270a8f None admin] RBAC: Authorizing `identity:get_user(user_id=c8d29301d2f84fcdb72d19727075628d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:32.947353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460edae-89bc-455c-aa1d-ce95ba270a8f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:32.949247 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 80/149] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:32 2022] GET /identity/v3/users/c8d29301d2f84fcdb72d19727075628d => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:32.954955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460edae-89bc-455c-aa1d-ce95ba270a8f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:33.024524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:33.036351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 70/150] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:33.037778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6731e-0fb3-4fd5-8ea0-b3696cfd582d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:33.062173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:33.190911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460edae-89bc-455c-aa1d-ce95ba270a8f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:33.276620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e56f0-f839-4faa-ba34-b309cd7f8dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:33.277239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e56f0-f839-4faa-ba34-b309cd7f8dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:33.277514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e56f0-f839-4faa-ba34-b309cd7f8dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:33.277781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e56f0-f839-4faa-ba34-b309cd7f8dd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:33.289125 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6731e-0fb3-4fd5-8ea0-b3696cfd582d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:33.373736 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b6d99-97cd-4f6a-85c5-800f3e7d8f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:33.374423 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b6d99-97cd-4f6a-85c5-800f3e7d8f8c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:33.374716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b6d99-97cd-4f6a-85c5-800f3e7d8f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:33.374990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b6d99-97cd-4f6a-85c5-800f3e7d8f8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:33.407410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b6d99-97cd-4f6a-85c5-800f3e7d8f8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-2c6b6d99-97cd-4f6a-85c5-800f3e7d8f8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:33.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:33.413229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:33.414004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:33.414004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:33.414004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:33.414004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:33.414004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:33.414004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:33.414004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:33.414004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:33.414004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:33.414004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:33.414004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:33.414004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:33.414004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 71/151] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:10:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:33.430949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e56f0-f839-4faa-ba34-b309cd7f8dd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:33.432442 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e56f0-f839-4faa-ba34-b309cd7f8dd1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:33.437342 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b6d99-97cd-4f6a-85c5-800f3e7d8f8c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:33.558201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:33.575487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:33.640320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 81/152] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 686 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:33.641970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd7295-37e0-431b-a50c-4cb2d6e6826e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:33.642481 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd7295-37e0-431b-a50c-4cb2d6e6826e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:33.642961 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd7295-37e0-431b-a50c-4cb2d6e6826e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:33.644828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 82/153] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:33.645765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd7295-37e0-431b-a50c-4cb2d6e6826e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:33.654102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b6d99-97cd-4f6a-85c5-800f3e7d8f8c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:33.699993 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20309a-150e-40e8-98b5-0e2b32475352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:33.700617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20309a-150e-40e8-98b5-0e2b32475352 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:33.700879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20309a-150e-40e8-98b5-0e2b32475352 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:33.701154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20309a-150e-40e8-98b5-0e2b32475352 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:33.701852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20309a-150e-40e8-98b5-0e2b32475352 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:33.702179 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20309a-150e-40e8-98b5-0e2b32475352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:33.703601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20309a-150e-40e8-98b5-0e2b32475352 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:33.719160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 72/154] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:33.720451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa4dd-a270-46d3-8d24-3173a9ef6435 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:33.720734 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa4dd-a270-46d3-8d24-3173a9ef6435 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:33.721005 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa4dd-a270-46d3-8d24-3173a9ef6435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:33.772078 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-054fa4dd-a270-46d3-8d24-3173a9ef6435 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:33.773340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-054fa4dd-a270-46d3-8d24-3173a9ef6435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:33.866304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd7295-37e0-431b-a50c-4cb2d6e6826e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:33.924781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb2a1d-e235-4628-8d03-186e344d12f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:33.925833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2a1d-e235-4628-8d03-186e344d12f6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:33.926359 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2a1d-e235-4628-8d03-186e344d12f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:33.926838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb2a1d-e235-4628-8d03-186e344d12f6 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:33.928789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb2a1d-e235-4628-8d03-186e344d12f6 None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:33.930491 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb2a1d-e235-4628-8d03-186e344d12f6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:33.932433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 83/155] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:33 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:33.933471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb2a1d-e235-4628-8d03-186e344d12f6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:33.939424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 73/156] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:33.940499 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fa4dd-a270-46d3-8d24-3173a9ef6435 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:33.966993 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:34.031280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:34.124258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb2a1d-e235-4628-8d03-186e344d12f6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:34.141044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fa4dd-a270-46d3-8d24-3173a9ef6435 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:34.181838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f3ee4-b2e7-4558-a70c-0b9800319167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:34.182861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f3ee4-b2e7-4558-a70c-0b9800319167 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:34.183281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f3ee4-b2e7-4558-a70c-0b9800319167 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:34.183694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f3ee4-b2e7-4558-a70c-0b9800319167 None admin] PATH_INFO: `/v3/users/60f027a86a574cfdbd746a543980fbdf` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:34.190096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f3ee4-b2e7-4558-a70c-0b9800319167 None admin] RBAC: Authorizing `identity:get_user(user_id=60f027a86a574cfdbd746a543980fbdf)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:34.191919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f3ee4-b2e7-4558-a70c-0b9800319167 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:34.194236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 84/157] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:33 2022] GET /identity/v3/users/60f027a86a574cfdbd746a543980fbdf => generated 247 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:34.195257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f3ee4-b2e7-4558-a70c-0b9800319167 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:34.217560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50014d-d9e0-40a3-bbbd-e5cbc8fe6597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:34.218405 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50014d-d9e0-40a3-bbbd-e5cbc8fe6597 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:34.218777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50014d-d9e0-40a3-bbbd-e5cbc8fe6597 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:34.219143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50014d-d9e0-40a3-bbbd-e5cbc8fe6597 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:34.249978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d50014d-d9e0-40a3-bbbd-e5cbc8fe6597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-2d50014d-d9e0-40a3-bbbd-e5cbc8fe6597 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:34.266982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:34.268527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:34.269824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:34.269824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:34.269824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:34.269824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:34.269824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:34.269824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:34.269824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:34.269824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:34.269824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:34.269824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:34.269824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:34.269824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:34.273018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 74/158] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:34.274840 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50014d-d9e0-40a3-bbbd-e5cbc8fe6597 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:34.354458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:34.358617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f3ee4-b2e7-4558-a70c-0b9800319167 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:34.394558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8960ae1-c6fc-42fb-989c-a22522e855ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:34.395488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8960ae1-c6fc-42fb-989c-a22522e855ec None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:34.395916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8960ae1-c6fc-42fb-989c-a22522e855ec None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:34.396965 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8960ae1-c6fc-42fb-989c-a22522e855ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:34.398011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8960ae1-c6fc-42fb-989c-a22522e855ec None admin] RBAC: Adding query filter params (role.id=c135afb53eaf43d9a99716160f0ffd25, scope.project.id=00cafd6359074bd5ae371a50effe7297, user.id=c8d29301d2f84fcdb72d19727075628d) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:34.398678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8960ae1-c6fc-42fb-989c-a22522e855ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:34.400484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8960ae1-c6fc-42fb-989c-a22522e855ec None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:34.407339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 85/159] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:34 2022] GET /identity/v3/role_assignments?user.id=c8d29301d2f84fcdb72d19727075628d&scope.project.id=00cafd6359074bd5ae371a50effe7297&role.id=c135afb53eaf43d9a99716160f0ffd25 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:34.409169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc883a-11d6-4a87-9acf-c4249aaf2d5d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:34.409609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc883a-11d6-4a87-9acf-c4249aaf2d5d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:34.410035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cc883a-11d6-4a87-9acf-c4249aaf2d5d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:34.411448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 86/160] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:34.412385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc883a-11d6-4a87-9acf-c4249aaf2d5d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:34.463550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50014d-d9e0-40a3-bbbd-e5cbc8fe6597 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:34.483434 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:34.540157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cfa42-5a17-4d1d-9882-6710d5314a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:34.541543 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cfa42-5a17-4d1d-9882-6710d5314a42 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:34.542198 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cfa42-5a17-4d1d-9882-6710d5314a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:34.542656 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cfa42-5a17-4d1d-9882-6710d5314a42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:34.572715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cc883a-11d6-4a87-9acf-c4249aaf2d5d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:34.611812 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b36a3-08bd-4828-974d-762ffbb260bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:34.612802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b36a3-08bd-4828-974d-762ffbb260bd None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:34.613275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b36a3-08bd-4828-974d-762ffbb260bd None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:34.613818 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b36a3-08bd-4828-974d-762ffbb260bd None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:34.614820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b36a3-08bd-4828-974d-762ffbb260bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:34.615428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b36a3-08bd-4828-974d-762ffbb260bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:34.617484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590b36a3-08bd-4828-974d-762ffbb260bd None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:34.627567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 87/161] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:34.629316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32c959-9e75-414b-90b8-fc25616753c5 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:34.629924 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32c959-9e75-414b-90b8-fc25616753c5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:34.630445 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32c959-9e75-414b-90b8-fc25616753c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:34.654787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8a32c959-9e75-414b-90b8-fc25616753c5 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:34.656511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8a32c959-9e75-414b-90b8-fc25616753c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:34.666358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cfa42-5a17-4d1d-9882-6710d5314a42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:34.666358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062cfa42-5a17-4d1d-9882-6710d5314a42 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:34.673027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:34.778210 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 88/162] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:34.779065 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32c959-9e75-414b-90b8-fc25616753c5 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:34.815838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 75/163] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:34.816715 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cfa42-5a17-4d1d-9882-6710d5314a42 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:34.841046 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:34.933791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32c959-9e75-414b-90b8-fc25616753c5 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:34.973411 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23151da4-ced6-45bd-b5b9-fdd0ab10df34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:34.975272 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23151da4-ced6-45bd-b5b9-fdd0ab10df34 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:34.975856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23151da4-ced6-45bd-b5b9-fdd0ab10df34 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:34.976338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23151da4-ced6-45bd-b5b9-fdd0ab10df34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:34.988211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23151da4-ced6-45bd-b5b9-fdd0ab10df34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-23151da4-ced6-45bd-b5b9-fdd0ab10df34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:34.993639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:34.994991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:34.995778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:34.995778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:34.995778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:34.995778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:34.995778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:34.995778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:34.995778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:34.995778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:34.995778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:34.995778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:34.995778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:34.995778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:34.996364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062cfa42-5a17-4d1d-9882-6710d5314a42 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:34.998348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 89/164] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:34.999396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23151da4-ced6-45bd-b5b9-fdd0ab10df34 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:35.026438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:35.054843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93350d-ac90-4836-a583-6e655af13f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:35.055495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93350d-ac90-4836-a583-6e655af13f28 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:35.055785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93350d-ac90-4836-a583-6e655af13f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:35.056078 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93350d-ac90-4836-a583-6e655af13f28 None admin] PATH_INFO: `/v3/users/2e0d857d7ed04548a601eba89bcc3d8e` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:35.057287 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93350d-ac90-4836-a583-6e655af13f28 None admin] RBAC: Authorizing `identity:get_user(user_id=2e0d857d7ed04548a601eba89bcc3d8e)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:35.058846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93350d-ac90-4836-a583-6e655af13f28 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:35.060689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 76/165] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:34 2022] GET /identity/v3/users/2e0d857d7ed04548a601eba89bcc3d8e => generated 289 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:35.061536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93350d-ac90-4836-a583-6e655af13f28 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:35.113857 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23151da4-ced6-45bd-b5b9-fdd0ab10df34 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:35.150827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac342fe-aa49-416e-bf57-ef99649ff819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:35.151831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac342fe-aa49-416e-bf57-ef99649ff819 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:35.152464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac342fe-aa49-416e-bf57-ef99649ff819 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:35.152948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac342fe-aa49-416e-bf57-ef99649ff819 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:35.154317 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac342fe-aa49-416e-bf57-ef99649ff819 None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:35.155964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac342fe-aa49-416e-bf57-ef99649ff819 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:35.158192 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 90/166] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:34 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:35.159160 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac342fe-aa49-416e-bf57-ef99649ff819 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:35.175989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:35.211706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93350d-ac90-4836-a583-6e655af13f28 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:35.237018 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:35.268184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bc987-7bb0-42e7-a8ed-4616b0c4da75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:35.268802 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bc987-7bb0-42e7-a8ed-4616b0c4da75 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:35.269082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bc987-7bb0-42e7-a8ed-4616b0c4da75 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:35.269351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bc987-7bb0-42e7-a8ed-4616b0c4da75 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:35.270023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bc987-7bb0-42e7-a8ed-4616b0c4da75 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:35.270350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bc987-7bb0-42e7-a8ed-4616b0c4da75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:35.271442 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac342fe-aa49-416e-bf57-ef99649ff819 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:35.271790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bc987-7bb0-42e7-a8ed-4616b0c4da75 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:35.285783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 77/167] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:35.286444 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bc987-7bb0-42e7-a8ed-4616b0c4da75 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:35.317937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65976f0-e3b8-4763-9c67-50896e4137ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:35.319325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65976f0-e3b8-4763-9c67-50896e4137ff None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:35.319766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65976f0-e3b8-4763-9c67-50896e4137ff None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:35.320178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65976f0-e3b8-4763-9c67-50896e4137ff None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:35.321489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65976f0-e3b8-4763-9c67-50896e4137ff None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:35.323180 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65976f0-e3b8-4763-9c67-50896e4137ff None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:35.324996 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 91/168] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:35 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:35.325932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65976f0-e3b8-4763-9c67-50896e4137ff None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:35.430463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65976f0-e3b8-4763-9c67-50896e4137ff None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:35.449806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bc987-7bb0-42e7-a8ed-4616b0c4da75 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:35.466543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19de8b-87e9-4cca-9279-87fc4e570da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:35.467391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19de8b-87e9-4cca-9279-87fc4e570da9 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:35.467832 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19de8b-87e9-4cca-9279-87fc4e570da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:35.468243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19de8b-87e9-4cca-9279-87fc4e570da9 None admin] PATH_INFO: `/v3/users/c8d29301d2f84fcdb72d19727075628d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:35.469528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19de8b-87e9-4cca-9279-87fc4e570da9 None admin] RBAC: Authorizing `identity:get_user(user_id=c8d29301d2f84fcdb72d19727075628d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:35.471217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19de8b-87e9-4cca-9279-87fc4e570da9 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:35.473089 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 92/169] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:35 2022] GET /identity/v3/users/c8d29301d2f84fcdb72d19727075628d => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:35.473967 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19de8b-87e9-4cca-9279-87fc4e570da9 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:35.483649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:35.511692 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebfbe8-e57f-48ce-a1a7-5cf2d554a1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:35.512309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebfbe8-e57f-48ce-a1a7-5cf2d554a1fe None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:35.512590 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebfbe8-e57f-48ce-a1a7-5cf2d554a1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:35.512863 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebfbe8-e57f-48ce-a1a7-5cf2d554a1fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:35.513602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebfbe8-e57f-48ce-a1a7-5cf2d554a1fe None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=00cafd6359074bd5ae371a50effe7297, user.id=60f027a86a574cfdbd746a543980fbdf) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:35.513909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebfbe8-e57f-48ce-a1a7-5cf2d554a1fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:35.515393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebfbe8-e57f-48ce-a1a7-5cf2d554a1fe None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:35.529318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 78/170] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:35 2022] GET /identity/v3/role_assignments?user.id=60f027a86a574cfdbd746a543980fbdf&scope.project.id=00cafd6359074bd5ae371a50effe7297&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:35.529955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebfbe8-e57f-48ce-a1a7-5cf2d554a1fe None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:35.538420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:35.620507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19de8b-87e9-4cca-9279-87fc4e570da9 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:35.680372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a81a5-956c-4074-96e2-7ca22b795ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:35.681095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a81a5-956c-4074-96e2-7ca22b795ea7 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:35.681380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a81a5-956c-4074-96e2-7ca22b795ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:35.681650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a81a5-956c-4074-96e2-7ca22b795ea7 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297/users/2e0d857d7ed04548a601eba89bcc3d8e/roles/a002aab559bb446593ca855874db44a2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:35.683767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a81a5-956c-4074-96e2-7ca22b795ea7 None admin] RBAC: Authorizing `identity:create_grant(project_id=00cafd6359074bd5ae371a50effe7297, user_id=2e0d857d7ed04548a601eba89bcc3d8e, role_id=a002aab559bb446593ca855874db44a2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:35.685230 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a81a5-956c-4074-96e2-7ca22b795ea7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:35.707890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 93/171] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:10:35 2022] PUT /identity/v3/projects/00cafd6359074bd5ae371a50effe7297/users/2e0d857d7ed04548a601eba89bcc3d8e/roles/a002aab559bb446593ca855874db44a2 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:10:35.709115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a81a5-956c-4074-96e2-7ca22b795ea7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:35.765514 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebfbe8-e57f-48ce-a1a7-5cf2d554a1fe None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:35.832362 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805eb1f-ca4a-4811-9485-0d6f565acba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:35.833032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805eb1f-ca4a-4811-9485-0d6f565acba1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:35.833274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805eb1f-ca4a-4811-9485-0d6f565acba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:35.833577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805eb1f-ca4a-4811-9485-0d6f565acba1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:35.856083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805eb1f-ca4a-4811-9485-0d6f565acba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b805eb1f-ca4a-4811-9485-0d6f565acba1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:35.861867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:35.862909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:35.863688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:35.863688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:35.863688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:35.863688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:35.863688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:35.863688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:35.863688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:35.863688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:35.863688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:35.863688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:35.863688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:10:35.863688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:35.872051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 79/172] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:10:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:35.877702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805eb1f-ca4a-4811-9485-0d6f565acba1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:35.909231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:35.968922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a81a5-956c-4074-96e2-7ca22b795ea7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:36.013585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa9174-31c6-469f-b273-e4c4c57418f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:36.014314 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:36.014629 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa9174-31c6-469f-b273-e4c4c57418f5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:36.014912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa9174-31c6-469f-b273-e4c4c57418f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:36.015217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa9174-31c6-469f-b273-e4c4c57418f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:36.062854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805eb1f-ca4a-4811-9485-0d6f565acba1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:36.103837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912afbc4-5a80-4e39-a74a-93c17087a88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:36.104458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912afbc4-5a80-4e39-a74a-93c17087a88e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:36.104731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912afbc4-5a80-4e39-a74a-93c17087a88e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:36.104994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912afbc4-5a80-4e39-a74a-93c17087a88e None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:36.105676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912afbc4-5a80-4e39-a74a-93c17087a88e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:36.105972 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912afbc4-5a80-4e39-a74a-93c17087a88e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:36.108984 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912afbc4-5a80-4e39-a74a-93c17087a88e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:36.118422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 80/173] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:10:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:36.119812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d294da-5f4a-4278-86ae-4158f86b34d1 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:36.120097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d294da-5f4a-4278-86ae-4158f86b34d1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:36.120371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d294da-5f4a-4278-86ae-4158f86b34d1 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:36.121699 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 81/174] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:36.122379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa9174-31c6-469f-b273-e4c4c57418f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:36.124199 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa9174-31c6-469f-b273-e4c4c57418f5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:36.127081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d294da-5f4a-4278-86ae-4158f86b34d1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:36.196507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:36.250353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 94/175] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:36.251638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680219ff-6994-45ab-a483-c8e8acf45424 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:36.251921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680219ff-6994-45ab-a483-c8e8acf45424 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:36.252192 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680219ff-6994-45ab-a483-c8e8acf45424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:36.270256 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d294da-5f4a-4278-86ae-4158f86b34d1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:36.272025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-680219ff-6994-45ab-a483-c8e8acf45424 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:36.273239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-680219ff-6994-45ab-a483-c8e8acf45424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:36.304940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:36.316419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991aec6a-84ea-434d-9274-ffc564e36aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:36.317873 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aec6a-84ea-434d-9274-ffc564e36aeb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:36.318448 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aec6a-84ea-434d-9274-ffc564e36aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:36.318880 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aec6a-84ea-434d-9274-ffc564e36aeb None admin] PATH_INFO: `/v3/users/c8d29301d2f84fcdb72d19727075628d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:36.320226 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991aec6a-84ea-434d-9274-ffc564e36aeb None admin] RBAC: Authorizing `identity:get_user(user_id=c8d29301d2f84fcdb72d19727075628d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:36.323430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991aec6a-84ea-434d-9274-ffc564e36aeb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:36.325603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 82/176] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:36 2022] GET /identity/v3/users/c8d29301d2f84fcdb72d19727075628d => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:36.327237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226edd9-6fcf-41ac-8740-6c67cffeca92 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:36.327667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226edd9-6fcf-41ac-8740-6c67cffeca92 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:36.328084 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226edd9-6fcf-41ac-8740-6c67cffeca92 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:36.329676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 83/177] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:36.331087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226edd9-6fcf-41ac-8740-6c67cffeca92 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:36.358151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 95/178] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:36.359330 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680219ff-6994-45ab-a483-c8e8acf45424 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:36.453999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226edd9-6fcf-41ac-8740-6c67cffeca92 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:36.486867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:36.490306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ccc5d2-695f-47f0-9e72-d607d6c79057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:36.490954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccc5d2-695f-47f0-9e72-d607d6c79057 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:36.491231 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccc5d2-695f-47f0-9e72-d607d6c79057 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:36.491479 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ccc5d2-695f-47f0-9e72-d607d6c79057 None admin] PATH_INFO: `/v3/projects/dd9619e9978a4e7b8ae0de623e4aebff` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:36.492612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccc5d2-695f-47f0-9e72-d607d6c79057 None admin] RBAC: Authorizing `identity:get_project(project_id=dd9619e9978a4e7b8ae0de623e4aebff)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:36.494337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ccc5d2-695f-47f0-9e72-d607d6c79057 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:36.495926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 84/179] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:36 2022] GET /identity/v3/projects/dd9619e9978a4e7b8ae0de623e4aebff => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:36.496694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680219ff-6994-45ab-a483-c8e8acf45424 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:36.501863 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f9a04-c8d3-4a79-877f-7fac8243d22b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:36.502169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f9a04-c8d3-4a79-877f-7fac8243d22b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:36.502445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f9a04-c8d3-4a79-877f-7fac8243d22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:36.524023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9b9f9a04-c8d3-4a79-877f-7fac8243d22b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:36.525271 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9b9f9a04-c8d3-4a79-877f-7fac8243d22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:36.536743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602252f6-e3cd-468a-92ba-791f6528d94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:36.537657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602252f6-e3cd-468a-92ba-791f6528d94e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:36.538083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602252f6-e3cd-468a-92ba-791f6528d94e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:36.538552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602252f6-e3cd-468a-92ba-791f6528d94e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:36.597400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:36.611579 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602252f6-e3cd-468a-92ba-791f6528d94e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:36.612729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 85/180] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:36.613624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602252f6-e3cd-468a-92ba-791f6528d94e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:36.614136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc63ab-8fe9-4ca6-803a-caba481ccd74 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:36.614419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc63ab-8fe9-4ca6-803a-caba481ccd74 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:36.614689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc63ab-8fe9-4ca6-803a-caba481ccd74 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:36.618128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 86/181] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:36.619005 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc63ab-8fe9-4ca6-803a-caba481ccd74 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:36.725903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc63ab-8fe9-4ca6-803a-caba481ccd74 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:36.747999 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:36.757480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 96/182] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:36.763461 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae39e6-f7d3-484f-b84e-a8cca3f16d0e None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:36.763795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae39e6-f7d3-484f-b84e-a8cca3f16d0e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:36.764015 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae39e6-f7d3-484f-b84e-a8cca3f16d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:36.767746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2707517-3067-4a1a-a5ac-6e09611ce8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:36.768735 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2707517-3067-4a1a-a5ac-6e09611ce8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:36.769159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2707517-3067-4a1a-a5ac-6e09611ce8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:36.769564 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2707517-3067-4a1a-a5ac-6e09611ce8c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:36.770611 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2707517-3067-4a1a-a5ac-6e09611ce8c6 None admin] RBAC: Adding query filter params (role.id=c135afb53eaf43d9a99716160f0ffd25, scope.project.id=dd9619e9978a4e7b8ae0de623e4aebff, user.id=c8d29301d2f84fcdb72d19727075628d) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:36.771254 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2707517-3067-4a1a-a5ac-6e09611ce8c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:36.772950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2707517-3067-4a1a-a5ac-6e09611ce8c6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:36.779982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 87/183] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:36 2022] GET /identity/v3/role_assignments?user.id=c8d29301d2f84fcdb72d19727075628d&scope.project.id=dd9619e9978a4e7b8ae0de623e4aebff&role.id=c135afb53eaf43d9a99716160f0ffd25 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:36.780933 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2707517-3067-4a1a-a5ac-6e09611ce8c6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:36.791328 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0cae39e6-f7d3-484f-b84e-a8cca3f16d0e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:36.792547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0cae39e6-f7d3-484f-b84e-a8cca3f16d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:36.848929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:36.904535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2707517-3067-4a1a-a5ac-6e09611ce8c6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:36.947225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dccae-d231-4eab-b3a9-32e6b4624a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:36.948116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dccae-d231-4eab-b3a9-32e6b4624a88 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:36.948550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dccae-d231-4eab-b3a9-32e6b4624a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:36.948977 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dccae-d231-4eab-b3a9-32e6b4624a88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:36.963498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dccae-d231-4eab-b3a9-32e6b4624a88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-746dccae-d231-4eab-b3a9-32e6b4624a88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:36.970922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:36.972128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:36.973155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:36.973155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:36.973155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:36.973155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:36.973155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:36.973155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:36.973155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:36.973155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:36.973155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:36.973155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:36.973155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:36.973155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:36.973794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 97/184] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:36.973794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae39e6-f7d3-484f-b84e-a8cca3f16d0e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:36.975894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 88/185] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:10:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:36.976919 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dccae-d231-4eab-b3a9-32e6b4624a88 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:37.101534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dccae-d231-4eab-b3a9-32e6b4624a88 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:37.103069 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae39e6-f7d3-484f-b84e-a8cca3f16d0e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:37.123320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:37.138920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:37.151163 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff77d6-c091-4a74-8911-07a9805c04d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:37.151769 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff77d6-c091-4a74-8911-07a9805c04d5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:37.152043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff77d6-c091-4a74-8911-07a9805c04d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:37.152314 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff77d6-c091-4a74-8911-07a9805c04d5 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:37.152999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9dc2c-abbd-4074-8432-36c294fab60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:37.153428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff77d6-c091-4a74-8911-07a9805c04d5 None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:37.153913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9dc2c-abbd-4074-8432-36c294fab60d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:37.154365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9dc2c-abbd-4074-8432-36c294fab60d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:37.154784 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9dc2c-abbd-4074-8432-36c294fab60d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:37.154931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff77d6-c091-4a74-8911-07a9805c04d5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:37.157088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 98/186] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:36 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:37.158308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff77d6-c091-4a74-8911-07a9805c04d5 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:37.176344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9dc2c-abbd-4074-8432-36c294fab60d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-45e9dc2c-abbd-4074-8432-36c294fab60d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:37.182518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:37.183624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:37.184535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:37.184535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:37.184535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:37.184535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:37.184535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:37.184535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:37.184535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:37.184535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:37.184535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:37.184535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:37.184535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:10:37.184535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:37.188697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 89/187] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:10:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:37.189821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9dc2c-abbd-4074-8432-36c294fab60d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:37.303565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff77d6-c091-4a74-8911-07a9805c04d5 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:37.314307 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9dc2c-abbd-4074-8432-36c294fab60d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:37.346364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd331f-0cba-4a14-a0d9-4ae43131b1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:37.346976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd331f-0cba-4a14-a0d9-4ae43131b1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:37.347280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd331f-0cba-4a14-a0d9-4ae43131b1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:37.347516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd331f-0cba-4a14-a0d9-4ae43131b1e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:37.348227 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd331f-0cba-4a14-a0d9-4ae43131b1e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:37.348484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd331f-0cba-4a14-a0d9-4ae43131b1e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:37.349931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dd331f-0cba-4a14-a0d9-4ae43131b1e1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:37.350570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad8839-246e-4c39-9e58-7b3eea47eb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:37.351284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad8839-246e-4c39-9e58-7b3eea47eb13 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:37.351617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad8839-246e-4c39-9e58-7b3eea47eb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:37.352116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad8839-246e-4c39-9e58-7b3eea47eb13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:37.352658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad8839-246e-4c39-9e58-7b3eea47eb13 None admin] RBAC: Adding query filter params (role.id=dfb71501dfed4ce48cf8cb3559cee9d6, scope.project.id=00cafd6359074bd5ae371a50effe7297, user.id=c8d29301d2f84fcdb72d19727075628d) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:37.352960 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad8839-246e-4c39-9e58-7b3eea47eb13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:37.354398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad8839-246e-4c39-9e58-7b3eea47eb13 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:37.359223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 99/188] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:37.366108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 90/189] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:37 2022] GET /identity/v3/role_assignments?user.id=c8d29301d2f84fcdb72d19727075628d&scope.project.id=00cafd6359074bd5ae371a50effe7297&role.id=dfb71501dfed4ce48cf8cb3559cee9d6 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:37.366108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad8839-246e-4c39-9e58-7b3eea47eb13 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:37.366349 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd331f-0cba-4a14-a0d9-4ae43131b1e1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:37.416012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:37.443544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:37.491810 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad8839-246e-4c39-9e58-7b3eea47eb13 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:37.496983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd331f-0cba-4a14-a0d9-4ae43131b1e1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:37.531458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a472d3e-8d10-4d63-8600-bd5711657ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:37.532290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a472d3e-8d10-4d63-8600-bd5711657ede None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:37.532667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a472d3e-8d10-4d63-8600-bd5711657ede None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:37.533035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a472d3e-8d10-4d63-8600-bd5711657ede None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:37.534297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a472d3e-8d10-4d63-8600-bd5711657ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:37.536804 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a472d3e-8d10-4d63-8600-bd5711657ede None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:37.539579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 91/190] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:37.540404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a472d3e-8d10-4d63-8600-bd5711657ede None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:37.543832 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c118c-66a8-4803-9bd1-20518fbe81c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:37.544707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c118c-66a8-4803-9bd1-20518fbe81c8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:37.545618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c118c-66a8-4803-9bd1-20518fbe81c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:37.546113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c118c-66a8-4803-9bd1-20518fbe81c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:37.547056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c118c-66a8-4803-9bd1-20518fbe81c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:37.547659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c118c-66a8-4803-9bd1-20518fbe81c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:37.549351 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c118c-66a8-4803-9bd1-20518fbe81c8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:37.558500 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 100/191] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:10:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:37.559634 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c118c-66a8-4803-9bd1-20518fbe81c8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:37.684326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a472d3e-8d10-4d63-8600-bd5711657ede None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:37.685114 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c118c-66a8-4803-9bd1-20518fbe81c8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:37.693881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:37.708057 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:37.720421 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e1383-4e20-4b43-ad35-6584e83690a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:37.721045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e1383-4e20-4b43-ad35-6584e83690a5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:37.721500 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e1383-4e20-4b43-ad35-6584e83690a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:37.721612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e1383-4e20-4b43-ad35-6584e83690a5 None admin] PATH_INFO: `/v3/users/30ad1066511b487cb12ba705d1c2f371` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:37.727064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f4ece-5126-46c4-b5fe-303435bb9fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:37.727669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f4ece-5126-46c4-b5fe-303435bb9fde None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:37.727945 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f4ece-5126-46c4-b5fe-303435bb9fde None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:37.728241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f4ece-5126-46c4-b5fe-303435bb9fde None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:37.728900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f4ece-5126-46c4-b5fe-303435bb9fde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:37.730836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f4ece-5126-46c4-b5fe-303435bb9fde None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:37.735915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e1383-4e20-4b43-ad35-6584e83690a5 None admin] RBAC: Authorizing `identity:get_user(user_id=30ad1066511b487cb12ba705d1c2f371)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:37.737482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3e1383-4e20-4b43-ad35-6584e83690a5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:37.744826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 92/192] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:37 2022] GET /identity/v3/users/30ad1066511b487cb12ba705d1c2f371 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:37.744826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e1383-4e20-4b43-ad35-6584e83690a5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:37.755181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 101/193] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:10:37 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:10:37.755181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f4ece-5126-46c4-b5fe-303435bb9fde None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:37.884783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f4ece-5126-46c4-b5fe-303435bb9fde None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:37.908995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e1383-4e20-4b43-ad35-6584e83690a5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:37.923722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571c0fb-1a59-48b3-bcd7-41417c07a684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:37.924487 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571c0fb-1a59-48b3-bcd7-41417c07a684 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:37.924873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571c0fb-1a59-48b3-bcd7-41417c07a684 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:37.925245 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571c0fb-1a59-48b3-bcd7-41417c07a684 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:37.959542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371e7a7-b73e-43eb-b042-6650a47c5f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:37.959911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371e7a7-b73e-43eb-b042-6650a47c5f0e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:37.961450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371e7a7-b73e-43eb-b042-6650a47c5f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:37.961450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371e7a7-b73e-43eb-b042-6650a47c5f0e None admin] PATH_INFO: `/v3/users/30ad1066511b487cb12ba705d1c2f371` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:37.962796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371e7a7-b73e-43eb-b042-6650a47c5f0e None admin] RBAC: Authorizing `identity:get_user(user_id=30ad1066511b487cb12ba705d1c2f371)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:37.964253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371e7a7-b73e-43eb-b042-6650a47c5f0e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:37.966022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 93/194] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:37 2022] GET /identity/v3/users/30ad1066511b487cb12ba705d1c2f371 => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:37.971673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371e7a7-b73e-43eb-b042-6650a47c5f0e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:37.979799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:38.017632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571c0fb-1a59-48b3-bcd7-41417c07a684 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:38.019426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571c0fb-1a59-48b3-bcd7-41417c07a684 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:38.021168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:38.112229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371e7a7-b73e-43eb-b042-6650a47c5f0e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:38.154869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 102/195] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:38.157233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68b7ce-b91b-471e-a0bc-640f72e6b9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:38.157834 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68b7ce-b91b-471e-a0bc-640f72e6b9ea None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:38.158831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68b7ce-b91b-471e-a0bc-640f72e6b9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:38.158831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68b7ce-b91b-471e-a0bc-640f72e6b9ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:38.160067 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571c0fb-1a59-48b3-bcd7-41417c07a684 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:38.253912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68b7ce-b91b-471e-a0bc-640f72e6b9ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:38.256022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68b7ce-b91b-471e-a0bc-640f72e6b9ea None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:38.264955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:38.299430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571c0fb-1a59-48b3-bcd7-41417c07a684 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:38.301183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:38.339120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845caf6-0d9c-467d-818e-485ac4a41b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:38.340121 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845caf6-0d9c-467d-818e-485ac4a41b59 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:38.340402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845caf6-0d9c-467d-818e-485ac4a41b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:38.340671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845caf6-0d9c-467d-818e-485ac4a41b59 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:38.342787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845caf6-0d9c-467d-818e-485ac4a41b59 None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:38.344253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845caf6-0d9c-467d-818e-485ac4a41b59 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:38.345900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 103/196] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:38 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:38.347053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5cc84-ab3c-4db3-9eff-ca283705cbfa None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:38.347331 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5cc84-ab3c-4db3-9eff-ca283705cbfa None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:38.347660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5cc84-ab3c-4db3-9eff-ca283705cbfa None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:38.348859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 104/197] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:38.353974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5cc84-ab3c-4db3-9eff-ca283705cbfa None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:38.384283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 94/198] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:38.385377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699270e0-cfad-4517-a891-f795d475acca None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:38.385657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699270e0-cfad-4517-a891-f795d475acca None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:38.385926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699270e0-cfad-4517-a891-f795d475acca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:38.409263 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-699270e0-cfad-4517-a891-f795d475acca None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:38.410544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-699270e0-cfad-4517-a891-f795d475acca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:38.481666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5cc84-ab3c-4db3-9eff-ca283705cbfa None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:38.481854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 95/199] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:38.482889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699270e0-cfad-4517-a891-f795d475acca None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:38.549131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d95fcc-73b0-4d47-9f98-aca9298e20ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:38.550128 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d95fcc-73b0-4d47-9f98-aca9298e20ba None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:38.550613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d95fcc-73b0-4d47-9f98-aca9298e20ba None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:38.551103 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d95fcc-73b0-4d47-9f98-aca9298e20ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:38.552036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d95fcc-73b0-4d47-9f98-aca9298e20ba None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=a9f733493ca84fbf8da49edbef53b518, user.id=30ad1066511b487cb12ba705d1c2f371) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:38.553075 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d95fcc-73b0-4d47-9f98-aca9298e20ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:38.554781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d95fcc-73b0-4d47-9f98-aca9298e20ba None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:38.561495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 105/200] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:38 2022] GET /identity/v3/role_assignments?user.id=30ad1066511b487cb12ba705d1c2f371&scope.project.id=a9f733493ca84fbf8da49edbef53b518&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:38.594926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:38.645136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699270e0-cfad-4517-a891-f795d475acca None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:38.690470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a0ead-c263-4556-9273-534aa0a74e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:38.691129 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a0ead-c263-4556-9273-534aa0a74e53 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:38.691404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a0ead-c263-4556-9273-534aa0a74e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:38.691674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a0ead-c263-4556-9273-534aa0a74e53 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:38.693483 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a0ead-c263-4556-9273-534aa0a74e53 None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:38.694973 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86a0ead-c263-4556-9273-534aa0a74e53 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:38.696671 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 96/201] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:38 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:38.702594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d95fcc-73b0-4d47-9f98-aca9298e20ba None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:38.719176 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:38.833939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d95fcc-73b0-4d47-9f98-aca9298e20ba None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:38.863360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a0ead-c263-4556-9273-534aa0a74e53 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:38.876797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f1961-eeff-4df0-b239-bb0da46f140e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:38.877654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f1961-eeff-4df0-b239-bb0da46f140e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:38.878099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f1961-eeff-4df0-b239-bb0da46f140e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:38.878513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f1961-eeff-4df0-b239-bb0da46f140e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:38.879486 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f1961-eeff-4df0-b239-bb0da46f140e None admin] RBAC: Adding query filter params (role.id=dfb71501dfed4ce48cf8cb3559cee9d6, scope.project.id=a9f733493ca84fbf8da49edbef53b518, user.id=30ad1066511b487cb12ba705d1c2f371) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:38.880089 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f1961-eeff-4df0-b239-bb0da46f140e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:38.882385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f1961-eeff-4df0-b239-bb0da46f140e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:38.894571 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 106/202] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:38 2022] GET /identity/v3/role_assignments?user.id=30ad1066511b487cb12ba705d1c2f371&scope.project.id=a9f733493ca84fbf8da49edbef53b518&role.id=dfb71501dfed4ce48cf8cb3559cee9d6 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:38.998930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a0ead-c263-4556-9273-534aa0a74e53 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:39.046312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:39.054186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8abe43-5d97-419e-820a-40229786ea14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:39.055307 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8abe43-5d97-419e-820a-40229786ea14 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:39.055918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8abe43-5d97-419e-820a-40229786ea14 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:39.056532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8abe43-5d97-419e-820a-40229786ea14 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:39.072464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8abe43-5d97-419e-820a-40229786ea14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1a8abe43-5d97-419e-820a-40229786ea14 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:39.077395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:39.078446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:39.079312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:39.079312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:39.079312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:39.079312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:39.079312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:39.079312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:39.079312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:39.079312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:39.079312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:39.079312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:39.079312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:39.079312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:39.079312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 97/203] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:10:39.084744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f1961-eeff-4df0-b239-bb0da46f140e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:39.169745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:39.209966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f1961-eeff-4df0-b239-bb0da46f140e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:39.245931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099bb3b-c7b7-4b8b-a8fb-4d16f858303e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:39.246566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099bb3b-c7b7-4b8b-a8fb-4d16f858303e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:39.246844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099bb3b-c7b7-4b8b-a8fb-4d16f858303e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:39.247112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099bb3b-c7b7-4b8b-a8fb-4d16f858303e None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:39.247773 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099bb3b-c7b7-4b8b-a8fb-4d16f858303e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:39.248125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099bb3b-c7b7-4b8b-a8fb-4d16f858303e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:39.249565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099bb3b-c7b7-4b8b-a8fb-4d16f858303e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:39.257689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 107/204] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:39.263521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8abe43-5d97-419e-820a-40229786ea14 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:39.374413 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8abe43-5d97-419e-820a-40229786ea14 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:39.429231 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0d060-2b3d-46e3-abfb-e196f3a947e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:39.429876 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0d060-2b3d-46e3-abfb-e196f3a947e5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:39.430174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0d060-2b3d-46e3-abfb-e196f3a947e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:39.430449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0d060-2b3d-46e3-abfb-e196f3a947e5 None admin] PATH_INFO: `/v3/users/2e0d857d7ed04548a601eba89bcc3d8e` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:39.432459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0d060-2b3d-46e3-abfb-e196f3a947e5 None admin] RBAC: Authorizing `identity:get_user(user_id=2e0d857d7ed04548a601eba89bcc3d8e)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:39.442095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0d060-2b3d-46e3-abfb-e196f3a947e5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:39.442095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 98/205] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:39 2022] GET /identity/v3/users/2e0d857d7ed04548a601eba89bcc3d8e => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:39.451661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099bb3b-c7b7-4b8b-a8fb-4d16f858303e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:39.660334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099bb3b-c7b7-4b8b-a8fb-4d16f858303e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:39.718425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:39.728273 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4765d-794e-4163-8e7a-0fed2084aad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:39.728905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4765d-794e-4163-8e7a-0fed2084aad1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:39.729195 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4765d-794e-4163-8e7a-0fed2084aad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:39.729473 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4765d-794e-4163-8e7a-0fed2084aad1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:39.901161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4765d-794e-4163-8e7a-0fed2084aad1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:39.902691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4765d-794e-4163-8e7a-0fed2084aad1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:40.030591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 108/206] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:40.032157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0d060-2b3d-46e3-abfb-e196f3a947e5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:40.098476 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:40.143208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0d060-2b3d-46e3-abfb-e196f3a947e5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:40.178715 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657b26a-27bb-4745-9e88-a687342c7cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:40.179346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657b26a-27bb-4745-9e88-a687342c7cdb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:40.179623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657b26a-27bb-4745-9e88-a687342c7cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:40.179891 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657b26a-27bb-4745-9e88-a687342c7cdb None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:40.181026 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657b26a-27bb-4745-9e88-a687342c7cdb None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:40.182494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1657b26a-27bb-4745-9e88-a687342c7cdb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:40.184433 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 99/207] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:40 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:40.190316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4765d-794e-4163-8e7a-0fed2084aad1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:40.225745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:40.328383 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4765d-794e-4163-8e7a-0fed2084aad1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:40.377785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade39371-7e67-4a3d-a528-fefbb96ded58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:40.378421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade39371-7e67-4a3d-a528-fefbb96ded58 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:40.378758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade39371-7e67-4a3d-a528-fefbb96ded58 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:40.379030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade39371-7e67-4a3d-a528-fefbb96ded58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:40.379815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade39371-7e67-4a3d-a528-fefbb96ded58 None admin] RBAC: Adding query filter params (role.id=a002aab559bb446593ca855874db44a2, scope.project.id=00cafd6359074bd5ae371a50effe7297, user.id=2e0d857d7ed04548a601eba89bcc3d8e) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:40.380663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade39371-7e67-4a3d-a528-fefbb96ded58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:40.382121 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade39371-7e67-4a3d-a528-fefbb96ded58 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:40.388675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 109/208] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:40 2022] GET /identity/v3/role_assignments?user.id=2e0d857d7ed04548a601eba89bcc3d8e&scope.project.id=00cafd6359074bd5ae371a50effe7297&role.id=a002aab559bb446593ca855874db44a2 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:40.673119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ef89a-76d7-411d-8d7e-ab88462ab97a None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:40.673508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ef89a-76d7-411d-8d7e-ab88462ab97a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:40.673706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ef89a-76d7-411d-8d7e-ab88462ab97a None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:40.675288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 100/209] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:40.683262 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bec7d-902c-4604-96a9-e69f79764edd None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:40.683894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bec7d-902c-4604-96a9-e69f79764edd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:40.684405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bec7d-902c-4604-96a9-e69f79764edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:40.706265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3e9bec7d-902c-4604-96a9-e69f79764edd None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:40.707830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3e9bec7d-902c-4604-96a9-e69f79764edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:40.788809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 110/210] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:40.816024 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c58a82-d3c8-42e4-b859-8c0560ef72f8 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:40.816328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c58a82-d3c8-42e4-b859-8c0560ef72f8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:40.816567 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c58a82-d3c8-42e4-b859-8c0560ef72f8 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:40.818194 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 101/211] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:40.823548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f4b15-dda5-4b3e-8907-c692baedce86 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:40.823824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f4b15-dda5-4b3e-8907-c692baedce86 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:40.824091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f4b15-dda5-4b3e-8907-c692baedce86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:40.846427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-426f4b15-dda5-4b3e-8907-c692baedce86 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:40.847612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-426f4b15-dda5-4b3e-8907-c692baedce86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:40.873810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:40.932822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 111/212] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:41.076305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c58a82-d3c8-42e4-b859-8c0560ef72f8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:41.181363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c58a82-d3c8-42e4-b859-8c0560ef72f8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:41.203047 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f4b15-dda5-4b3e-8907-c692baedce86 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:41.212910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:41.223441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864f302-c4b1-4a98-b5b0-93145694cab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:41.224702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864f302-c4b1-4a98-b5b0-93145694cab5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:41.224981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864f302-c4b1-4a98-b5b0-93145694cab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:41.225249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864f302-c4b1-4a98-b5b0-93145694cab5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:41.238769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864f302-c4b1-4a98-b5b0-93145694cab5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-e864f302-c4b1-4a98-b5b0-93145694cab5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:41.246212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:41.247123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:41.247908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:41.247908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:41.247908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:41.247908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:41.247908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:41.247908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:41.247908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:41.247908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:41.247908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:41.247908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:41.247908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:41.247908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:41.247908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 102/213] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:41.254362 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864f302-c4b1-4a98-b5b0-93145694cab5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:41.339021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f4b15-dda5-4b3e-8907-c692baedce86 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:41.376611 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864f302-c4b1-4a98-b5b0-93145694cab5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:41.381220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ece802-8237-42d9-a060-990d61184b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:41.382320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ece802-8237-42d9-a060-990d61184b5f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:41.382743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ece802-8237-42d9-a060-990d61184b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:41.383173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ece802-8237-42d9-a060-990d61184b5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:41.384511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ece802-8237-42d9-a060-990d61184b5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:41.386788 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ece802-8237-42d9-a060-990d61184b5f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:41.389566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 112/214] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:41.397617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ece802-8237-42d9-a060-990d61184b5f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:41.416534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d25b4e-dfff-41e5-8700-ae29db6bd65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:41.417142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d25b4e-dfff-41e5-8700-ae29db6bd65c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:41.417421 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d25b4e-dfff-41e5-8700-ae29db6bd65c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:41.417697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d25b4e-dfff-41e5-8700-ae29db6bd65c None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:41.418398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d25b4e-dfff-41e5-8700-ae29db6bd65c None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:41.419148 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d25b4e-dfff-41e5-8700-ae29db6bd65c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:41.420212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d25b4e-dfff-41e5-8700-ae29db6bd65c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:41.422699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:41.432083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 103/215] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:41.437882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d25b4e-dfff-41e5-8700-ae29db6bd65c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:41.490622 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:41.516022 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ece802-8237-42d9-a060-990d61184b5f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:41.548031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d25b4e-dfff-41e5-8700-ae29db6bd65c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:41.549864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591a627-47af-49b3-95d3-0f4a04c3d247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:41.550503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591a627-47af-49b3-95d3-0f4a04c3d247 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:41.550778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591a627-47af-49b3-95d3-0f4a04c3d247 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:41.551131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591a627-47af-49b3-95d3-0f4a04c3d247 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:41.551762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591a627-47af-49b3-95d3-0f4a04c3d247 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:41.553716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591a627-47af-49b3-95d3-0f4a04c3d247 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:41.578768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 113/216] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:10:41 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:10:41.580025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4892c-eb07-4999-8141-c92ecf779bae None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:41.580310 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4892c-eb07-4999-8141-c92ecf779bae None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:41.580583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4892c-eb07-4999-8141-c92ecf779bae None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:41.581948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 114/217] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:41.587401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aac660-7f9c-4bc4-8410-1ac72e7a9359 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:41.587689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aac660-7f9c-4bc4-8410-1ac72e7a9359 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:41.587968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aac660-7f9c-4bc4-8410-1ac72e7a9359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:41.602521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf750cc-8871-4a67-9145-ea008f1b8386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:41.603462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf750cc-8871-4a67-9145-ea008f1b8386 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:41.603939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf750cc-8871-4a67-9145-ea008f1b8386 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:41.604521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf750cc-8871-4a67-9145-ea008f1b8386 None admin] PATH_INFO: `/v3/users/e488658a09594272b20a5bf954380192` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:41.607088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-92aac660-7f9c-4bc4-8410-1ac72e7a9359 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:41.608454 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-92aac660-7f9c-4bc4-8410-1ac72e7a9359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:41.625448 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf750cc-8871-4a67-9145-ea008f1b8386 None admin] RBAC: Authorizing `identity:get_user(user_id=e488658a09594272b20a5bf954380192)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:41.626983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf750cc-8871-4a67-9145-ea008f1b8386 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:41.628921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 104/218] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:41 2022] GET /identity/v3/users/e488658a09594272b20a5bf954380192 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:41.635095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf750cc-8871-4a67-9145-ea008f1b8386 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:41.699031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 115/219] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:41.759877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf750cc-8871-4a67-9145-ea008f1b8386 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:41.773348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:41.801314 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375df083-86d0-441b-a714-483588aa8195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:41.803272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375df083-86d0-441b-a714-483588aa8195 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:41.803582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375df083-86d0-441b-a714-483588aa8195 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:41.803879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375df083-86d0-441b-a714-483588aa8195 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:41.834780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083759fa-7d9c-404b-94eb-e9026ca1e7d1 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:41.835062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083759fa-7d9c-404b-94eb-e9026ca1e7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:41.835339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083759fa-7d9c-404b-94eb-e9026ca1e7d1 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:41.836928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 116/220] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:10:41.841774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe2439-1d29-466b-8dae-13c078184a59 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:41.842061 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe2439-1d29-466b-8dae-13c078184a59 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:41.842384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe2439-1d29-466b-8dae-13c078184a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:41.867480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e5fe2439-1d29-466b-8dae-13c078184a59 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:41.868664 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e5fe2439-1d29-466b-8dae-13c078184a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:41.872784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:41.880848 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375df083-86d0-441b-a714-483588aa8195 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:41.882359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375df083-86d0-441b-a714-483588aa8195 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:41.956842 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 117/221] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:41.974734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe2439-1d29-466b-8dae-13c078184a59 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:42.012822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 105/222] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 27 17:10:42.018450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375df083-86d0-441b-a714-483588aa8195 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:42.030296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:42.095920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe2439-1d29-466b-8dae-13c078184a59 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:42.136976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc557db1-cf0a-475e-8cdf-f1206bc825e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:42.136976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc557db1-cf0a-475e-8cdf-f1206bc825e7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:42.136976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc557db1-cf0a-475e-8cdf-f1206bc825e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:42.136976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc557db1-cf0a-475e-8cdf-f1206bc825e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:42.151151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375df083-86d0-441b-a714-483588aa8195 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:42.157739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:42.167340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc557db1-cf0a-475e-8cdf-f1206bc825e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-fc557db1-cf0a-475e-8cdf-f1206bc825e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:42.172152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:42.173051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:42.174851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:42.174851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:42.174851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:42.174851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:42.174851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:42.174851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:42.174851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:42.174851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:42.174851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:42.174851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:42.174851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:42.174851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:42.174851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 118/223] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:10:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:42.183059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc557db1-cf0a-475e-8cdf-f1206bc825e7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:42.225572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11957267-16b9-423f-920b-c7c259b751fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:42.226232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11957267-16b9-423f-920b-c7c259b751fa None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:42.226510 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11957267-16b9-423f-920b-c7c259b751fa None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:42.226780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11957267-16b9-423f-920b-c7c259b751fa None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:42.228571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11957267-16b9-423f-920b-c7c259b751fa None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:42.230010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11957267-16b9-423f-920b-c7c259b751fa None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:42.231874 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 106/224] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:42 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:42.238482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11957267-16b9-423f-920b-c7c259b751fa None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:42.346463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc557db1-cf0a-475e-8cdf-f1206bc825e7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:42.353049 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:42.379743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11957267-16b9-423f-920b-c7c259b751fa None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:42.394219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde31d2-3f77-43d7-8784-5c8ea15d11ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:42.395167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde31d2-3f77-43d7-8784-5c8ea15d11ba None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:42.396448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde31d2-3f77-43d7-8784-5c8ea15d11ba None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:42.397421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde31d2-3f77-43d7-8784-5c8ea15d11ba None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:42.398647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde31d2-3f77-43d7-8784-5c8ea15d11ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:42.399437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde31d2-3f77-43d7-8784-5c8ea15d11ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:42.401475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde31d2-3f77-43d7-8784-5c8ea15d11ba None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:42.413540 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 119/225] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:42.414790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde31d2-3f77-43d7-8784-5c8ea15d11ba None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:42.422294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb1579-d3d8-4c53-8679-e09daf1fb02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:42.422903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb1579-d3d8-4c53-8679-e09daf1fb02d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:42.423211 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb1579-d3d8-4c53-8679-e09daf1fb02d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:42.423540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb1579-d3d8-4c53-8679-e09daf1fb02d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:42.424270 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb1579-d3d8-4c53-8679-e09daf1fb02d None admin] RBAC: Adding query filter params (role.id=c135afb53eaf43d9a99716160f0ffd25, scope.project.id=a9f733493ca84fbf8da49edbef53b518, user.id=e488658a09594272b20a5bf954380192) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:42.424568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb1579-d3d8-4c53-8679-e09daf1fb02d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:42.425987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb1579-d3d8-4c53-8679-e09daf1fb02d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:42.438299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 107/226] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:42 2022] GET /identity/v3/role_assignments?user.id=e488658a09594272b20a5bf954380192&scope.project.id=a9f733493ca84fbf8da49edbef53b518&role.id=c135afb53eaf43d9a99716160f0ffd25 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:42.440016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb1579-d3d8-4c53-8679-e09daf1fb02d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:42.499711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:42.547003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde31d2-3f77-43d7-8784-5c8ea15d11ba None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:42.584756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f9335-8f87-488e-bd6f-c7ecb1d3fc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:42.585764 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f9335-8f87-488e-bd6f-c7ecb1d3fc2e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:42.586280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f9335-8f87-488e-bd6f-c7ecb1d3fc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:42.586768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f9335-8f87-488e-bd6f-c7ecb1d3fc2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:42.617118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f9335-8f87-488e-bd6f-c7ecb1d3fc2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-1b7f9335-8f87-488e-bd6f-c7ecb1d3fc2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:42.622522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:42.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:42.624205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:42.624205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:42.624205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:42.624205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:42.624205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:42.624205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:42.624205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:42.624205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:42.624205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:42.624205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:42.624205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:10:42.624205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:42.624205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 120/227] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:10:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:42.629522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f9335-8f87-488e-bd6f-c7ecb1d3fc2e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:42.660134 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb1579-d3d8-4c53-8679-e09daf1fb02d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:42.696584 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df9813-ecca-4317-809e-f9137ac86c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:42.697467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df9813-ecca-4317-809e-f9137ac86c64 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:42.697809 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df9813-ecca-4317-809e-f9137ac86c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:42.698175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df9813-ecca-4317-809e-f9137ac86c64 None admin] PATH_INFO: `/v3/users/30ad1066511b487cb12ba705d1c2f371` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:42.699398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df9813-ecca-4317-809e-f9137ac86c64 None admin] RBAC: Authorizing `identity:get_user(user_id=30ad1066511b487cb12ba705d1c2f371)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:42.701006 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df9813-ecca-4317-809e-f9137ac86c64 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:42.702988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 108/228] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:42 2022] GET /identity/v3/users/30ad1066511b487cb12ba705d1c2f371 => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:42.709113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df9813-ecca-4317-809e-f9137ac86c64 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:42.726398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:42.753880 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f9335-8f87-488e-bd6f-c7ecb1d3fc2e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:42.779953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:42.805190 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c78ae4-52b0-4538-812c-47f37afcd1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:42.805801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c78ae4-52b0-4538-812c-47f37afcd1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:42.806095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c78ae4-52b0-4538-812c-47f37afcd1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:42.806421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c78ae4-52b0-4538-812c-47f37afcd1a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:42.807093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c78ae4-52b0-4538-812c-47f37afcd1a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:42.808488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c78ae4-52b0-4538-812c-47f37afcd1a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:42.809852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c78ae4-52b0-4538-812c-47f37afcd1a3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:42.830134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 121/229] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:10:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:42.835761 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c78ae4-52b0-4538-812c-47f37afcd1a3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:42.892424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df9813-ecca-4317-809e-f9137ac86c64 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:42.961190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41444-356f-48e7-96dc-280100d5280b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:42.961831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41444-356f-48e7-96dc-280100d5280b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:42.962119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41444-356f-48e7-96dc-280100d5280b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:42.962389 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41444-356f-48e7-96dc-280100d5280b None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:42.965511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f41444-356f-48e7-96dc-280100d5280b None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:42.967652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f41444-356f-48e7-96dc-280100d5280b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:42.969542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 109/230] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:42 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:42.976745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41444-356f-48e7-96dc-280100d5280b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:43.015814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c78ae4-52b0-4538-812c-47f37afcd1a3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:43.073363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1737268-4bbc-4d77-91fb-7c9e2e3fae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:43.073987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1737268-4bbc-4d77-91fb-7c9e2e3fae92 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:43.074353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1737268-4bbc-4d77-91fb-7c9e2e3fae92 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:43.074619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1737268-4bbc-4d77-91fb-7c9e2e3fae92 None admin] PATH_INFO: `/v3/users/30ad1066511b487cb12ba705d1c2f371` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:43.075785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1737268-4bbc-4d77-91fb-7c9e2e3fae92 None admin] RBAC: Authorizing `identity:get_user(user_id=30ad1066511b487cb12ba705d1c2f371)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:43.077218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1737268-4bbc-4d77-91fb-7c9e2e3fae92 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:43.080493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 122/231] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:42 2022] GET /identity/v3/users/30ad1066511b487cb12ba705d1c2f371 => generated 283 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:43.084913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1737268-4bbc-4d77-91fb-7c9e2e3fae92 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:43.115247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:43.159856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41444-356f-48e7-96dc-280100d5280b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:43.181651 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:43.222040 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148680a1-13d0-4208-9d58-9215c5bc8999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:43.222723 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148680a1-13d0-4208-9d58-9215c5bc8999 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:43.222996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148680a1-13d0-4208-9d58-9215c5bc8999 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:43.223264 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148680a1-13d0-4208-9d58-9215c5bc8999 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518/users/30ad1066511b487cb12ba705d1c2f371/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:43.228236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148680a1-13d0-4208-9d58-9215c5bc8999 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9f733493ca84fbf8da49edbef53b518, user_id=30ad1066511b487cb12ba705d1c2f371, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:43.229686 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148680a1-13d0-4208-9d58-9215c5bc8999 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:43.252199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 110/232] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:10:42 2022] PUT /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518/users/30ad1066511b487cb12ba705d1c2f371/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:10:43.265364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1737268-4bbc-4d77-91fb-7c9e2e3fae92 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:43.306106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3c4a2-a13e-47d7-9f69-65995f2e7f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:43.306106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3c4a2-a13e-47d7-9f69-65995f2e7f5e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:43.306106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3c4a2-a13e-47d7-9f69-65995f2e7f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:43.306106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3c4a2-a13e-47d7-9f69-65995f2e7f5e None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:43.312088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3c4a2-a13e-47d7-9f69-65995f2e7f5e None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:43.312088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3c4a2-a13e-47d7-9f69-65995f2e7f5e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:43.312088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 123/233] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:43 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:43.315868 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148680a1-13d0-4208-9d58-9215c5bc8999 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:43.454307 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148680a1-13d0-4208-9d58-9215c5bc8999 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:43.494655 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b01086-b2dc-4a7d-98c1-8cb10c2fe766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:43.495321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b01086-b2dc-4a7d-98c1-8cb10c2fe766 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:43.495594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b01086-b2dc-4a7d-98c1-8cb10c2fe766 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:43.495862 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b01086-b2dc-4a7d-98c1-8cb10c2fe766 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518/users/30ad1066511b487cb12ba705d1c2f371/roles/dfb71501dfed4ce48cf8cb3559cee9d6` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:43.498171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b01086-b2dc-4a7d-98c1-8cb10c2fe766 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9f733493ca84fbf8da49edbef53b518, user_id=30ad1066511b487cb12ba705d1c2f371, role_id=dfb71501dfed4ce48cf8cb3559cee9d6)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:43.499682 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b01086-b2dc-4a7d-98c1-8cb10c2fe766 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:43.503182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:43.529884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 111/234] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:10:43 2022] PUT /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518/users/30ad1066511b487cb12ba705d1c2f371/roles/dfb71501dfed4ce48cf8cb3559cee9d6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:10:44.514560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfb5ed-a1d7-4497-8546-6b6f1d28a335 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:44.515061 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfb5ed-a1d7-4497-8546-6b6f1d28a335 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:44.515135 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfb5ed-a1d7-4497-8546-6b6f1d28a335 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:44.516704 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 124/235] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:44.522409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64af91-9526-47cf-939a-78eb8d5f3288 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:44.522823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64af91-9526-47cf-939a-78eb8d5f3288 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:44.522996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d64af91-9526-47cf-939a-78eb8d5f3288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:44.547087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5d64af91-9526-47cf-939a-78eb8d5f3288 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:44.548278 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5d64af91-9526-47cf-939a-78eb8d5f3288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:44.613459 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a89fbf-7bc5-413c-ab3b-4d40ffb7ac1f None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:44.613790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a89fbf-7bc5-413c-ab3b-4d40ffb7ac1f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:44.614012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a89fbf-7bc5-413c-ab3b-4d40ffb7ac1f None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:44.616145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 125/236] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:44.627287 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 112/237] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:44.628868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d236d0-227c-46c6-b519-78ad2a6c6ad2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:44.629151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d236d0-227c-46c6-b519-78ad2a6c6ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:44.629426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d236d0-227c-46c6-b519-78ad2a6c6ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:44.667025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-50d236d0-227c-46c6-b519-78ad2a6c6ad2 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:44.668207 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-50d236d0-227c-46c6-b519-78ad2a6c6ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:44.756380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 126/238] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:44.906696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64af91-9526-47cf-939a-78eb8d5f3288 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:45.037474 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d64af91-9526-47cf-939a-78eb8d5f3288 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:45.078627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:45.088861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853324fb-213e-492f-a315-fd9df2486e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:45.089742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853324fb-213e-492f-a315-fd9df2486e34 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:45.090275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853324fb-213e-492f-a315-fd9df2486e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:45.090738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853324fb-213e-492f-a315-fd9df2486e34 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:45.101701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853324fb-213e-492f-a315-fd9df2486e34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-853324fb-213e-492f-a315-fd9df2486e34 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:45.106792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:45.107916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:45.108696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:45.108696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:45.108696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:45.108696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:45.108696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:45.108696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:45.108696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:45.108696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:45.108696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:45.108696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:45.108696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:45.108696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:45.109295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e69af-6a15-4414-8a73-271bacc202e1 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:45.109838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e69af-6a15-4414-8a73-271bacc202e1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:45.109838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e69af-6a15-4414-8a73-271bacc202e1 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:45.111065 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 127/239] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:10:45.111688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 113/240] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:45.117392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e69af-6a15-4414-8a73-271bacc202e1 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:45.127268 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa8853-f548-4d6d-8198-432aba5b81b1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:45.127794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa8853-f548-4d6d-8198-432aba5b81b1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:45.128259 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa8853-f548-4d6d-8198-432aba5b81b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:45.154224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-42fa8853-f548-4d6d-8198-432aba5b81b1 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:45.156456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-42fa8853-f548-4d6d-8198-432aba5b81b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:45.181419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:45.240217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 114/241] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:45.241601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa8853-f548-4d6d-8198-432aba5b81b1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:45.246060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e69af-6a15-4414-8a73-271bacc202e1 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:45.285931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d3ae1-5646-47d4-aeaf-810c2affa3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:45.287048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d3ae1-5646-47d4-aeaf-810c2affa3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:45.287991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d3ae1-5646-47d4-aeaf-810c2affa3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:45.288480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d3ae1-5646-47d4-aeaf-810c2affa3e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:45.289868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d3ae1-5646-47d4-aeaf-810c2affa3e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:45.290432 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d3ae1-5646-47d4-aeaf-810c2affa3e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:45.292271 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d3ae1-5646-47d4-aeaf-810c2affa3e6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:45.311005 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 128/242] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:45.318327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d3ae1-5646-47d4-aeaf-810c2affa3e6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:45.359163 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa8853-f548-4d6d-8198-432aba5b81b1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:45.377479 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:45.398786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03d9a9-a533-40f6-b9ad-00ac8445febf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:45.399506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03d9a9-a533-40f6-b9ad-00ac8445febf None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:45.400128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03d9a9-a533-40f6-b9ad-00ac8445febf None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:45.400128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03d9a9-a533-40f6-b9ad-00ac8445febf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:45.401416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03d9a9-a533-40f6-b9ad-00ac8445febf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:45.403178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03d9a9-a533-40f6-b9ad-00ac8445febf None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:45.405480 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 115/243] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:45.410979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03d9a9-a533-40f6-b9ad-00ac8445febf None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:45.475908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d3ae1-5646-47d4-aeaf-810c2affa3e6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:45.504491 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:45.547794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb48258-fa50-4e7b-af98-057345a223ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:45.549439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb48258-fa50-4e7b-af98-057345a223ac None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:45.551014 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb48258-fa50-4e7b-af98-057345a223ac None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:45.551484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb48258-fa50-4e7b-af98-057345a223ac None admin] PATH_INFO: `/v3/users/e1d2758336884ead9bb43c24b6f369f0` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:45.579138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb48258-fa50-4e7b-af98-057345a223ac None admin] RBAC: Authorizing `identity:get_user(user_id=e1d2758336884ead9bb43c24b6f369f0)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:45.583642 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb48258-fa50-4e7b-af98-057345a223ac None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:45.588416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 129/244] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:45 2022] GET /identity/v3/users/e1d2758336884ead9bb43c24b6f369f0 => generated 297 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:45.595647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb48258-fa50-4e7b-af98-057345a223ac None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:45.610316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03d9a9-a533-40f6-b9ad-00ac8445febf None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:45.645462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3876f0-b2be-465a-9060-fdd2d282135c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:45.646106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3876f0-b2be-465a-9060-fdd2d282135c None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:45.646387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3876f0-b2be-465a-9060-fdd2d282135c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:45.646653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3876f0-b2be-465a-9060-fdd2d282135c None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:45.647298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3876f0-b2be-465a-9060-fdd2d282135c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:45.648716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3876f0-b2be-465a-9060-fdd2d282135c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:45.678240 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 116/245] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:10:45 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:10:45.679222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3876f0-b2be-465a-9060-fdd2d282135c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:45.766816 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:45.776644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb48258-fa50-4e7b-af98-057345a223ac None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:45.846823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3876f0-b2be-465a-9060-fdd2d282135c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:45.862839 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db2493-24de-4a19-b6a8-fa5e4b6a866a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:45.863474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db2493-24de-4a19-b6a8-fa5e4b6a866a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:45.863757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db2493-24de-4a19-b6a8-fa5e4b6a866a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:45.864070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db2493-24de-4a19-b6a8-fa5e4b6a866a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:45.886161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83250-4f46-40bc-bbb8-1d419679306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:45.886801 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83250-4f46-40bc-bbb8-1d419679306c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:45.887075 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83250-4f46-40bc-bbb8-1d419679306c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:45.887368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d83250-4f46-40bc-bbb8-1d419679306c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:45.899854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d83250-4f46-40bc-bbb8-1d419679306c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-47d83250-4f46-40bc-bbb8-1d419679306c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:45.905727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:45.906748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:45.907546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:45.907546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:45.907546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:45.907546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:45.907546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:45.907546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:45.907546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:45.907546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:45.907546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:45.907546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:45.907546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:45.907546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:45.907546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 117/246] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:45.913599 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83250-4f46-40bc-bbb8-1d419679306c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:45.930380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:45.966282 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db2493-24de-4a19-b6a8-fa5e4b6a866a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:45.967765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db2493-24de-4a19-b6a8-fa5e4b6a866a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:46.069765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:46.074506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d83250-4f46-40bc-bbb8-1d419679306c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:46.090579 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 130/247] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 27 17:10:46.092282 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291177af-d2b9-470c-890e-96021ff703be None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:46.092767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291177af-d2b9-470c-890e-96021ff703be None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:46.093219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291177af-d2b9-470c-890e-96021ff703be None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:46.095143 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 131/248] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:46.095550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291177af-d2b9-470c-890e-96021ff703be None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:46.122557 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5a7be-40b5-4471-94a2-8b88b01619c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:46.123986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5a7be-40b5-4471-94a2-8b88b01619c5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:46.123986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5a7be-40b5-4471-94a2-8b88b01619c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:46.124172 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5a7be-40b5-4471-94a2-8b88b01619c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:46.124907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5a7be-40b5-4471-94a2-8b88b01619c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:46.125207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5a7be-40b5-4471-94a2-8b88b01619c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:46.127418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5a7be-40b5-4471-94a2-8b88b01619c5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:46.139771 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 118/249] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:46.141853 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a5aca-ea6a-4b31-a013-c7c475c3bd06 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:46.142177 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a5aca-ea6a-4b31-a013-c7c475c3bd06 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:46.142454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a5aca-ea6a-4b31-a013-c7c475c3bd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:46.163052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3d0a5aca-ea6a-4b31-a013-c7c475c3bd06 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:46.164245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3d0a5aca-ea6a-4b31-a013-c7c475c3bd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:46.202339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:46.250642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 119/250] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:46.250828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291177af-d2b9-470c-890e-96021ff703be None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:46.251971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a5aca-ea6a-4b31-a013-c7c475c3bd06 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:46.331763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aab9a0-3b5e-4cde-b541-3a2f8ad12964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:46.332335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aab9a0-3b5e-4cde-b541-3a2f8ad12964 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:46.332610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aab9a0-3b5e-4cde-b541-3a2f8ad12964 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:46.333320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aab9a0-3b5e-4cde-b541-3a2f8ad12964 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:46.334494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aab9a0-3b5e-4cde-b541-3a2f8ad12964 None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:46.335968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aab9a0-3b5e-4cde-b541-3a2f8ad12964 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:46.337628 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 132/251] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:46 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:46.343566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aab9a0-3b5e-4cde-b541-3a2f8ad12964 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:46.356085 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:46.367691 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a5aca-ea6a-4b31-a013-c7c475c3bd06 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:46.408515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ce337-1741-4a66-92dd-23c3f0e2243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:46.409577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ce337-1741-4a66-92dd-23c3f0e2243e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:46.410195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ce337-1741-4a66-92dd-23c3f0e2243e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:46.410727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ce337-1741-4a66-92dd-23c3f0e2243e None admin] PATH_INFO: `/v3/users/e488658a09594272b20a5bf954380192` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:46.413632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ce337-1741-4a66-92dd-23c3f0e2243e None admin] RBAC: Authorizing `identity:get_user(user_id=e488658a09594272b20a5bf954380192)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:46.415499 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ce337-1741-4a66-92dd-23c3f0e2243e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:46.417710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 120/252] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:46 2022] GET /identity/v3/users/e488658a09594272b20a5bf954380192 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:46.419167 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e871c34-26d6-498c-8fc2-bf017cc39926 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:46.419649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e871c34-26d6-498c-8fc2-bf017cc39926 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:46.420119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e871c34-26d6-498c-8fc2-bf017cc39926 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:46.421850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 121/253] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:46.426939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e871c34-26d6-498c-8fc2-bf017cc39926 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:46.491345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aab9a0-3b5e-4cde-b541-3a2f8ad12964 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:46.534060 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e871c34-26d6-498c-8fc2-bf017cc39926 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:46.566829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8754631-34ae-412b-821b-da65ddc8717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:46.568061 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8754631-34ae-412b-821b-da65ddc8717b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:46.568254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8754631-34ae-412b-821b-da65ddc8717b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:46.568474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8754631-34ae-412b-821b-da65ddc8717b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:46.569379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8754631-34ae-412b-821b-da65ddc8717b None admin] RBAC: Adding query filter params (role.id=a002aab559bb446593ca855874db44a2, scope.project.id=a9f733493ca84fbf8da49edbef53b518, user.id=e1d2758336884ead9bb43c24b6f369f0) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:46.569683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8754631-34ae-412b-821b-da65ddc8717b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:46.571356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8754631-34ae-412b-821b-da65ddc8717b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:46.572800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0ee68-4f8f-4079-93df-fbb1fa69ca64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:46.573815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0ee68-4f8f-4079-93df-fbb1fa69ca64 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:46.574318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0ee68-4f8f-4079-93df-fbb1fa69ca64 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:46.574801 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0ee68-4f8f-4079-93df-fbb1fa69ca64 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:46.576180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0ee68-4f8f-4079-93df-fbb1fa69ca64 None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:46.578149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c0ee68-4f8f-4079-93df-fbb1fa69ca64 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:46.580267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 122/254] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:46 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:46.581151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 133/255] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:46 2022] GET /identity/v3/role_assignments?user.id=e1d2758336884ead9bb43c24b6f369f0&scope.project.id=a9f733493ca84fbf8da49edbef53b518&role.id=a002aab559bb446593ca855874db44a2 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:46.581886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a9f37-e903-4b49-95c6-464782350a34 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:46.582105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8754631-34ae-412b-821b-da65ddc8717b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:46.586096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a9f37-e903-4b49-95c6-464782350a34 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:46.586646 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:46.587497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a9f37-e903-4b49-95c6-464782350a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:46.613921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d13a9f37-e903-4b49-95c6-464782350a34 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:46.616160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d13a9f37-e903-4b49-95c6-464782350a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:46.646898 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:46.691008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 123/256] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:46.692256 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a9f37-e903-4b49-95c6-464782350a34 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:46.812240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8754631-34ae-412b-821b-da65ddc8717b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:46.837926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a9f37-e903-4b49-95c6-464782350a34 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:46.876481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b12f5-eb7e-4e17-b5ad-a336aff6cd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:46.877652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b12f5-eb7e-4e17-b5ad-a336aff6cd25 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:46.878163 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b12f5-eb7e-4e17-b5ad-a336aff6cd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:46.878635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b12f5-eb7e-4e17-b5ad-a336aff6cd25 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518/users/e488658a09594272b20a5bf954380192/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:46.881429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b12f5-eb7e-4e17-b5ad-a336aff6cd25 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9f733493ca84fbf8da49edbef53b518, user_id=e488658a09594272b20a5bf954380192, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:46.883309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b12f5-eb7e-4e17-b5ad-a336aff6cd25 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:46.894796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c910b-ccbb-4964-a8f8-07eaafeadaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:46.896023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c910b-ccbb-4964-a8f8-07eaafeadaf8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:46.896391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c910b-ccbb-4964-a8f8-07eaafeadaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:46.896746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c910b-ccbb-4964-a8f8-07eaafeadaf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:46.906591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 124/257] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:10:46 2022] PUT /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518/users/e488658a09594272b20a5bf954380192/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:10:46.913388 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c910b-ccbb-4964-a8f8-07eaafeadaf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-4e8c910b-ccbb-4964-a8f8-07eaafeadaf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:46.919055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:46.919994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:46.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:46.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:46.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:46.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:46.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:46.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:46.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:46.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:46.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:46.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:46.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:10:46.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:46.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 134/258] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:10:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:46.927634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b12f5-eb7e-4e17-b5ad-a336aff6cd25 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:46.959989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:47.071803 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c910b-ccbb-4964-a8f8-07eaafeadaf8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:47.076102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b12f5-eb7e-4e17-b5ad-a336aff6cd25 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:47.121819 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c18e40-5fad-4ae6-80a4-355fd4cad897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:47.122837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c18e40-5fad-4ae6-80a4-355fd4cad897 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:47.123326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c18e40-5fad-4ae6-80a4-355fd4cad897 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:47.123793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c18e40-5fad-4ae6-80a4-355fd4cad897 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:47.124668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c18e40-5fad-4ae6-80a4-355fd4cad897 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:47.125853 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c18e40-5fad-4ae6-80a4-355fd4cad897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:47.127510 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c18e40-5fad-4ae6-80a4-355fd4cad897 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:47.138112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 125/259] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:47.149776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c18e40-5fad-4ae6-80a4-355fd4cad897 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:47.231618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:47.262604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:47.275281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c18e40-5fad-4ae6-80a4-355fd4cad897 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:47.313913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f857ab9-cf50-466c-9610-fc79a183efc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:47.314995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f857ab9-cf50-466c-9610-fc79a183efc9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:47.315486 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f857ab9-cf50-466c-9610-fc79a183efc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:47.315985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f857ab9-cf50-466c-9610-fc79a183efc9 None admin] PATH_INFO: `/v3/users/30ad1066511b487cb12ba705d1c2f371` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:47.317763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f857ab9-cf50-466c-9610-fc79a183efc9 None admin] RBAC: Authorizing `identity:get_user(user_id=30ad1066511b487cb12ba705d1c2f371)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:47.319598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f857ab9-cf50-466c-9610-fc79a183efc9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:47.321409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 126/260] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:47 2022] GET /identity/v3/users/30ad1066511b487cb12ba705d1c2f371 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:47.329099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f857ab9-cf50-466c-9610-fc79a183efc9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:47.336753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c910b-ccbb-4964-a8f8-07eaafeadaf8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:47.423840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d34cf-030f-4c1f-a51c-4b616163f502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:47.424401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d34cf-030f-4c1f-a51c-4b616163f502 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:47.424677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d34cf-030f-4c1f-a51c-4b616163f502 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:47.424945 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d34cf-030f-4c1f-a51c-4b616163f502 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:47.438096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4d34cf-030f-4c1f-a51c-4b616163f502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ec4d34cf-030f-4c1f-a51c-4b616163f502 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:47.443583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:47.444495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:47.445290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:47.445290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:47.445290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:47.445290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:47.445290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:47.445290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:47.445290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:47.445290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:47.445290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:47.445290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:47.445290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:10:47.445290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:47.445290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 135/261] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:10:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:47.455885 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d34cf-030f-4c1f-a51c-4b616163f502 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:47.525159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f857ab9-cf50-466c-9610-fc79a183efc9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:47.619093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593662f-c806-40a6-bdc4-e48e53a4fef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:47.620484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593662f-c806-40a6-bdc4-e48e53a4fef8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:47.620957 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593662f-c806-40a6-bdc4-e48e53a4fef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:47.621437 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593662f-c806-40a6-bdc4-e48e53a4fef8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:47.666095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:47.697496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d34cf-030f-4c1f-a51c-4b616163f502 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:47.738169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:47.770672 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593662f-c806-40a6-bdc4-e48e53a4fef8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:47.772144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593662f-c806-40a6-bdc4-e48e53a4fef8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:47.773961 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d27f9-0eb1-4c99-a752-434833bc2857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:47.774629 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d27f9-0eb1-4c99-a752-434833bc2857 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:47.774905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d27f9-0eb1-4c99-a752-434833bc2857 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:47.775168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2d27f9-0eb1-4c99-a752-434833bc2857 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:47.775830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d27f9-0eb1-4c99-a752-434833bc2857 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:47.776122 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d27f9-0eb1-4c99-a752-434833bc2857 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:47.777495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2d27f9-0eb1-4c99-a752-434833bc2857 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:47.804318 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 136/262] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:10:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:47.819484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d27f9-0eb1-4c99-a752-434833bc2857 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:47.913887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 127/263] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:47.919152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593662f-c806-40a6-bdc4-e48e53a4fef8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:47.990592 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:48.064905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593662f-c806-40a6-bdc4-e48e53a4fef8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:48.080006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2d27f9-0eb1-4c99-a752-434833bc2857 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:48.136566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a16dd-09b2-4a13-93b0-248ad361b91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:48.137210 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a16dd-09b2-4a13-93b0-248ad361b91a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:48.137489 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a16dd-09b2-4a13-93b0-248ad361b91a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:48.137759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476a16dd-09b2-4a13-93b0-248ad361b91a None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:48.138913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a16dd-09b2-4a13-93b0-248ad361b91a None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:48.140355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476a16dd-09b2-4a13-93b0-248ad361b91a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:48.142332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 128/264] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:47 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:48.148086 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a16dd-09b2-4a13-93b0-248ad361b91a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:48.158178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06291373-d246-4230-a9b5-79a94595ab3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:48.158879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06291373-d246-4230-a9b5-79a94595ab3e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:48.159158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06291373-d246-4230-a9b5-79a94595ab3e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:48.159425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06291373-d246-4230-a9b5-79a94595ab3e None admin] PATH_INFO: `/v3/users/30ad1066511b487cb12ba705d1c2f371` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:48.160614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06291373-d246-4230-a9b5-79a94595ab3e None admin] RBAC: Authorizing `identity:get_user(user_id=30ad1066511b487cb12ba705d1c2f371)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:48.162122 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06291373-d246-4230-a9b5-79a94595ab3e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:48.164036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 137/265] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:47 2022] GET /identity/v3/users/30ad1066511b487cb12ba705d1c2f371 => generated 283 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:48.169330 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06291373-d246-4230-a9b5-79a94595ab3e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:48.278302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476a16dd-09b2-4a13-93b0-248ad361b91a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:48.293979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:48.310502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:48.314797 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d1f13-b9b7-4855-85d0-49f121eee301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:48.315545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d1f13-b9b7-4855-85d0-49f121eee301 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:48.315927 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d1f13-b9b7-4855-85d0-49f121eee301 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:48.316305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d1f13-b9b7-4855-85d0-49f121eee301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:48.317583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d1f13-b9b7-4855-85d0-49f121eee301 None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=a9f733493ca84fbf8da49edbef53b518, user.id=30ad1066511b487cb12ba705d1c2f371) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:48.317726 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d1f13-b9b7-4855-85d0-49f121eee301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:48.319328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d1f13-b9b7-4855-85d0-49f121eee301 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:48.327340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 129/266] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:48 2022] GET /identity/v3/role_assignments?user.id=30ad1066511b487cb12ba705d1c2f371&scope.project.id=a9f733493ca84fbf8da49edbef53b518&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:48.390265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06291373-d246-4230-a9b5-79a94595ab3e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:48.453432 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbfeab-6bf6-4822-affb-7160353c2c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:48.454063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbfeab-6bf6-4822-affb-7160353c2c38 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:48.454360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbfeab-6bf6-4822-affb-7160353c2c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:48.454625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbfeab-6bf6-4822-affb-7160353c2c38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:48.538454 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedbfeab-6bf6-4822-affb-7160353c2c38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:48.540039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedbfeab-6bf6-4822-affb-7160353c2c38 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:48.622594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:48.684774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 138/267] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 27 17:10:48.690293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d1f13-b9b7-4855-85d0-49f121eee301 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:48.748577 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a41db4-cc30-4873-b293-0417977baaf9 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:48.749077 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a41db4-cc30-4873-b293-0417977baaf9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:48.749186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a41db4-cc30-4873-b293-0417977baaf9 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:48.750759 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 139/268] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:48.756086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa113e-4f82-4f8d-9a28-d2faf1132a84 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:48.756372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa113e-4f82-4f8d-9a28-d2faf1132a84 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:48.756649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa113e-4f82-4f8d-9a28-d2faf1132a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:48.787075 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5eaa113e-4f82-4f8d-9a28-d2faf1132a84 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:48.788281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5eaa113e-4f82-4f8d-9a28-d2faf1132a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:48.819209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d1f13-b9b7-4855-85d0-49f121eee301 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:48.857912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a382af-0ab1-4e28-88db-f08de881468b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:48.858606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a382af-0ab1-4e28-88db-f08de881468b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:48.858881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a382af-0ab1-4e28-88db-f08de881468b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:48.859154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a382af-0ab1-4e28-88db-f08de881468b None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:48.860273 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a382af-0ab1-4e28-88db-f08de881468b None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:48.861713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a382af-0ab1-4e28-88db-f08de881468b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:48.863391 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 130/269] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:48 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:48.869296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a382af-0ab1-4e28-88db-f08de881468b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:48.917664 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 140/270] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:48.922388 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:48.975613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a382af-0ab1-4e28-88db-f08de881468b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:49.011213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7a4abb-55a6-49e4-bdc8-1491cdf3fb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:49.011822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a4abb-55a6-49e4-bdc8-1491cdf3fb98 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:49.012098 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a4abb-55a6-49e4-bdc8-1491cdf3fb98 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:49.012368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7a4abb-55a6-49e4-bdc8-1491cdf3fb98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:49.013095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a4abb-55a6-49e4-bdc8-1491cdf3fb98 None admin] RBAC: Adding query filter params (role.id=dfb71501dfed4ce48cf8cb3559cee9d6, scope.project.id=a9f733493ca84fbf8da49edbef53b518, user.id=30ad1066511b487cb12ba705d1c2f371) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:49.013400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a4abb-55a6-49e4-bdc8-1491cdf3fb98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:49.014870 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7a4abb-55a6-49e4-bdc8-1491cdf3fb98 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:49.020888 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 131/271] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:48 2022] GET /identity/v3/role_assignments?user.id=30ad1066511b487cb12ba705d1c2f371&scope.project.id=a9f733493ca84fbf8da49edbef53b518&role.id=dfb71501dfed4ce48cf8cb3559cee9d6 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:49.338551 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa113e-4f82-4f8d-9a28-d2faf1132a84 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:49.429913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:49.456730 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa113e-4f82-4f8d-9a28-d2faf1132a84 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:49.461423 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f5784-966f-4d8a-af62-eaa4e5712877 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:49.461696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f5784-966f-4d8a-af62-eaa4e5712877 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:49.462262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f5784-966f-4d8a-af62-eaa4e5712877 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:49.464122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 132/272] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:49.469290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd1810-7348-4144-be13-cbc86bd71167 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:49.469831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd1810-7348-4144-be13-cbc86bd71167 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:49.470340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd1810-7348-4144-be13-cbc86bd71167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:49.496162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9a2ea-09a0-45c6-93a0-b9ac3553ec94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:49.496797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9a2ea-09a0-45c6-93a0-b9ac3553ec94 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:49.497456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9a2ea-09a0-45c6-93a0-b9ac3553ec94 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:49.497456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9a2ea-09a0-45c6-93a0-b9ac3553ec94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:49.498501 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-57dd1810-7348-4144-be13-cbc86bd71167 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:49.500118 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-57dd1810-7348-4144-be13-cbc86bd71167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:49.515985 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9a2ea-09a0-45c6-93a0-b9ac3553ec94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a9d9a2ea-09a0-45c6-93a0-b9ac3553ec94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:49.521576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:49.522588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:49.523401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:49.523401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:49.523401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:49.523401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:49.523401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:49.523401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:49.523401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:49.523401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:49.523401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:49.523401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:49.523401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:49.523401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:49.523401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 141/273] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:49.529098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9a2ea-09a0-45c6-93a0-b9ac3553ec94 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:49.605961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 133/274] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:49.653378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f8aab-65eb-4109-afac-bbfed7bdc199 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:49.654154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f8aab-65eb-4109-afac-bbfed7bdc199 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:49.654650 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f8aab-65eb-4109-afac-bbfed7bdc199 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:49.656233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 134/275] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:10:49.661555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45c79b-ff02-435a-803a-83d6e11d32f6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:49.661863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9a2ea-09a0-45c6-93a0-b9ac3553ec94 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:49.662355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45c79b-ff02-435a-803a-83d6e11d32f6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:49.662825 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45c79b-ff02-435a-803a-83d6e11d32f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:49.681873 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3d45c79b-ff02-435a-803a-83d6e11d32f6 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:49.683409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3d45c79b-ff02-435a-803a-83d6e11d32f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:49.698787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c6dba-bfb1-486f-8063-a0f0589437b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:49.699392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c6dba-bfb1-486f-8063-a0f0589437b0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:49.699727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c6dba-bfb1-486f-8063-a0f0589437b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:49.700142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c6dba-bfb1-486f-8063-a0f0589437b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:49.700818 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c6dba-bfb1-486f-8063-a0f0589437b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:49.701153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c6dba-bfb1-486f-8063-a0f0589437b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:49.702582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c6dba-bfb1-486f-8063-a0f0589437b0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:49.711535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 142/276] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:49.717226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c6dba-bfb1-486f-8063-a0f0589437b0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:49.721879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:49.727085 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:49.775516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 135/277] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:49.836688 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c6dba-bfb1-486f-8063-a0f0589437b0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:49.873476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f2cf6-615c-456e-8a6d-14dde86a2a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:49.874347 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f2cf6-615c-456e-8a6d-14dde86a2a26 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:49.874764 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f2cf6-615c-456e-8a6d-14dde86a2a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:49.875174 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f2cf6-615c-456e-8a6d-14dde86a2a26 None admin] PATH_INFO: `/v3/users/e70d2494392a435093c29ce3caac6296` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:49.890910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f2cf6-615c-456e-8a6d-14dde86a2a26 None admin] RBAC: Authorizing `identity:get_user(user_id=e70d2494392a435093c29ce3caac6296)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:49.892584 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f2cf6-615c-456e-8a6d-14dde86a2a26 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:49.894670 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 143/278] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:49 2022] GET /identity/v3/users/e70d2494392a435093c29ce3caac6296 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:49.900556 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45c79b-ff02-435a-803a-83d6e11d32f6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:49.904513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f2cf6-615c-456e-8a6d-14dde86a2a26 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:49.990300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:50.038830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45c79b-ff02-435a-803a-83d6e11d32f6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:50.053883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f2cf6-615c-456e-8a6d-14dde86a2a26 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:50.084742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b590c37-662d-43ce-ad5b-5373c2ecb883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:50.085346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b590c37-662d-43ce-ad5b-5373c2ecb883 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:50.085649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b590c37-662d-43ce-ad5b-5373c2ecb883 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:50.085889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b590c37-662d-43ce-ad5b-5373c2ecb883 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:50.086725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b590c37-662d-43ce-ad5b-5373c2ecb883 None admin] RBAC: Adding query filter params (role.id=c135afb53eaf43d9a99716160f0ffd25, user.id=e70d2494392a435093c29ce3caac6296, scope.system=all) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:50.086953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b590c37-662d-43ce-ad5b-5373c2ecb883 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:50.088363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b590c37-662d-43ce-ad5b-5373c2ecb883 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:50.093765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 136/279] 10.222.0.55 () {56 vars in 1308 bytes} [Tue Sep 27 17:10:49 2022] GET /identity/v3/role_assignments?user.id=e70d2494392a435093c29ce3caac6296&scope.system=all&role.id=c135afb53eaf43d9a99716160f0ffd25 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:50.095937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a2a62-5a81-4212-b980-f06bdb0678ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:50.096581 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a2a62-5a81-4212-b980-f06bdb0678ae None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:50.096856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a2a62-5a81-4212-b980-f06bdb0678ae None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:50.097126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a2a62-5a81-4212-b980-f06bdb0678ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:50.098860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a2a62-5a81-4212-b980-f06bdb0678ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:50.099947 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b590c37-662d-43ce-ad5b-5373c2ecb883 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:50.100338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a2a62-5a81-4212-b980-f06bdb0678ae None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:50.106631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 144/280] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:50.109563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a2a62-5a81-4212-b980-f06bdb0678ae None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:50.153045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:50.225704 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a2a62-5a81-4212-b980-f06bdb0678ae None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:50.228482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b590c37-662d-43ce-ad5b-5373c2ecb883 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:50.256563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:50.263456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae8c63-cb66-4494-a85f-a428618fbf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:50.264073 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae8c63-cb66-4494-a85f-a428618fbf0c None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:50.264352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae8c63-cb66-4494-a85f-a428618fbf0c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:50.264622 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cae8c63-cb66-4494-a85f-a428618fbf0c None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:50.265277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae8c63-cb66-4494-a85f-a428618fbf0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:50.266747 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cae8c63-cb66-4494-a85f-a428618fbf0c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:50.273051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a837a-7d5a-40f2-a6af-bbbc1ef73af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:50.273686 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a837a-7d5a-40f2-a6af-bbbc1ef73af2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:50.273970 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a837a-7d5a-40f2-a6af-bbbc1ef73af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:50.274268 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a837a-7d5a-40f2-a6af-bbbc1ef73af2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:50.288119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a837a-7d5a-40f2-a6af-bbbc1ef73af2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-bf1a837a-7d5a-40f2-a6af-bbbc1ef73af2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:50.293250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:50.294222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:50.295052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:50.295052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:50.295052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:50.295052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:50.295052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:50.295052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:50.295052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:50.295052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:50.295052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:50.295052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:50.295052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:50.295052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:50.295052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 137/281] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:50.296867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 145/282] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:10:50 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:10:50.300467 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae8c63-cb66-4494-a85f-a428618fbf0c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:50.300962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f6a8b-dfe9-4b0b-8071-022dbb4a8060 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:50.301302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f6a8b-dfe9-4b0b-8071-022dbb4a8060 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:50.301586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f6a8b-dfe9-4b0b-8071-022dbb4a8060 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:50.304552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 138/283] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:50.315047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5da9a-9947-4296-a9d2-b5d00a5e4c2c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:50.315047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5da9a-9947-4296-a9d2-b5d00a5e4c2c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:50.315047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5da9a-9947-4296-a9d2-b5d00a5e4c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:50.349502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c1e5da9a-9947-4296-a9d2-b5d00a5e4c2c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:50.351069 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c1e5da9a-9947-4296-a9d2-b5d00a5e4c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:50.442179 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 139/284] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:50.456310 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cae8c63-cb66-4494-a85f-a428618fbf0c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:50.494190 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d9ae9-e554-4226-a8ff-629c1fc50c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:50.494795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d9ae9-e554-4226-a8ff-629c1fc50c6d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:50.495071 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d9ae9-e554-4226-a8ff-629c1fc50c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:50.495343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d9ae9-e554-4226-a8ff-629c1fc50c6d None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:50.496061 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d9ae9-e554-4226-a8ff-629c1fc50c6d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:50.496365 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d9ae9-e554-4226-a8ff-629c1fc50c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:50.497729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d9ae9-e554-4226-a8ff-629c1fc50c6d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:50.506850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 146/285] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:50.512422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5da9a-9947-4296-a9d2-b5d00a5e4c2c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:50.548793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:50.619917 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5da9a-9947-4296-a9d2-b5d00a5e4c2c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:50.660563 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9a4c8-8c82-4155-a75e-7140b552da71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:50.661564 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9a4c8-8c82-4155-a75e-7140b552da71 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:50.662046 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9a4c8-8c82-4155-a75e-7140b552da71 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:50.662562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9a4c8-8c82-4155-a75e-7140b552da71 None admin] PATH_INFO: `/v3/users/e1d2758336884ead9bb43c24b6f369f0` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:50.663937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9a4c8-8c82-4155-a75e-7140b552da71 None admin] RBAC: Authorizing `identity:get_user(user_id=e1d2758336884ead9bb43c24b6f369f0)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:50.665563 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9a4c8-8c82-4155-a75e-7140b552da71 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:50.667703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 140/286] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:50 2022] GET /identity/v3/users/e1d2758336884ead9bb43c24b6f369f0 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:50.673077 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d9ae9-e554-4226-a8ff-629c1fc50c6d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:50.735510 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:50.752401 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9a4c8-8c82-4155-a75e-7140b552da71 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:50.799829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d9ae9-e554-4226-a8ff-629c1fc50c6d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:50.845223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03411c-1677-4871-8415-b82735851d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:50.846556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03411c-1677-4871-8415-b82735851d89 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:50.847205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03411c-1677-4871-8415-b82735851d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:50.849165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03411c-1677-4871-8415-b82735851d89 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:50.850558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03411c-1677-4871-8415-b82735851d89 None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:50.852148 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03411c-1677-4871-8415-b82735851d89 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:50.854871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 147/287] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:50 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:50.860702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03411c-1677-4871-8415-b82735851d89 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:50.896861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:50.910419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9a4c8-8c82-4155-a75e-7140b552da71 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:50.965952 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ebe2c-069d-4c52-8aaf-e3317f029e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:50.966854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ebe2c-069d-4c52-8aaf-e3317f029e21 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:50.967302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ebe2c-069d-4c52-8aaf-e3317f029e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:50.967729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ebe2c-069d-4c52-8aaf-e3317f029e21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:50.989976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ebe2c-069d-4c52-8aaf-e3317f029e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d14ebe2c-069d-4c52-8aaf-e3317f029e21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:50.995878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:50.997008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:50.997976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:50.997976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:50.997976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:50.997976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:50.997976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:50.997976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:50.997976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:50.997976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:50.997976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:50.997976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:50.997976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:50.997976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:51.005750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 141/288] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:51.015220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ebe2c-069d-4c52-8aaf-e3317f029e21 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:51.084266 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03411c-1677-4871-8415-b82735851d89 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:51.123833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:51.125645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ebe2c-069d-4c52-8aaf-e3317f029e21 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:51.161521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2397f2b-5466-4194-a1a5-04bb6d0c4111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:51.162515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2397f2b-5466-4194-a1a5-04bb6d0c4111 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:51.163031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2397f2b-5466-4194-a1a5-04bb6d0c4111 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:51.163508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2397f2b-5466-4194-a1a5-04bb6d0c4111 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:51.164377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2397f2b-5466-4194-a1a5-04bb6d0c4111 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:51.164900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2397f2b-5466-4194-a1a5-04bb6d0c4111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:51.166508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2397f2b-5466-4194-a1a5-04bb6d0c4111 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:51.175473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 142/289] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:51.181878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2397f2b-5466-4194-a1a5-04bb6d0c4111 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:51.198783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4684517-c716-4d2c-a70e-66d8e48916ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:51.199467 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4684517-c716-4d2c-a70e-66d8e48916ef None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:51.199747 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4684517-c716-4d2c-a70e-66d8e48916ef None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:51.200021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4684517-c716-4d2c-a70e-66d8e48916ef None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518/users/e1d2758336884ead9bb43c24b6f369f0/roles/a002aab559bb446593ca855874db44a2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:51.206818 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4684517-c716-4d2c-a70e-66d8e48916ef None admin] RBAC: Authorizing `identity:create_grant(project_id=a9f733493ca84fbf8da49edbef53b518, user_id=e1d2758336884ead9bb43c24b6f369f0, role_id=a002aab559bb446593ca855874db44a2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:51.207093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:51.208843 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4684517-c716-4d2c-a70e-66d8e48916ef None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:51.229678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 148/290] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:10:50 2022] PUT /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518/users/e1d2758336884ead9bb43c24b6f369f0/roles/a002aab559bb446593ca855874db44a2 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:10:51.293956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2397f2b-5466-4194-a1a5-04bb6d0c4111 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:51.331257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b796e6-9944-4eea-9bf7-9ce797fbc3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:51.331913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b796e6-9944-4eea-9bf7-9ce797fbc3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:51.332259 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b796e6-9944-4eea-9bf7-9ce797fbc3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:51.333072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b796e6-9944-4eea-9bf7-9ce797fbc3f4 None admin] PATH_INFO: `/v3/users/e488658a09594272b20a5bf954380192` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:51.334396 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b796e6-9944-4eea-9bf7-9ce797fbc3f4 None admin] RBAC: Authorizing `identity:get_user(user_id=e488658a09594272b20a5bf954380192)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:51.335854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b796e6-9944-4eea-9bf7-9ce797fbc3f4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:51.337465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 143/291] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:51 2022] GET /identity/v3/users/e488658a09594272b20a5bf954380192 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:51.343336 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4684517-c716-4d2c-a70e-66d8e48916ef None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:51.499113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4684517-c716-4d2c-a70e-66d8e48916ef None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:51.533585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc72b3c-5071-4991-8e7c-340cc9096c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:51.534327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc72b3c-5071-4991-8e7c-340cc9096c2a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:51.534606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc72b3c-5071-4991-8e7c-340cc9096c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:51.534922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc72b3c-5071-4991-8e7c-340cc9096c2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:51.582981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:51.606779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc72b3c-5071-4991-8e7c-340cc9096c2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:51.608547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc72b3c-5071-4991-8e7c-340cc9096c2a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:51.742785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 149/292] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:51.748457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b796e6-9944-4eea-9bf7-9ce797fbc3f4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:51.896696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:51.905814 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b796e6-9944-4eea-9bf7-9ce797fbc3f4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:51.959372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f2a64-d9ba-4785-9b08-9f1aa17828e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:51.960014 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f2a64-d9ba-4785-9b08-9f1aa17828e6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:51.960296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f2a64-d9ba-4785-9b08-9f1aa17828e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:51.960574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f2a64-d9ba-4785-9b08-9f1aa17828e6 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:51.961750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f2a64-d9ba-4785-9b08-9f1aa17828e6 None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:51.963226 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f2a64-d9ba-4785-9b08-9f1aa17828e6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:51.964875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 144/293] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:51 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:51.975163 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc72b3c-5071-4991-8e7c-340cc9096c2a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:52.094266 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:52.123783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc72b3c-5071-4991-8e7c-340cc9096c2a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:52.166873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d6bc3-bad3-408f-87be-0a9d0be0b375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:52.167409 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6bc3-bad3-408f-87be-0a9d0be0b375 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:52.167689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6bc3-bad3-408f-87be-0a9d0be0b375 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:52.168000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6bc3-bad3-408f-87be-0a9d0be0b375 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:52.168700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d6bc3-bad3-408f-87be-0a9d0be0b375 None admin] RBAC: Adding query filter params (role.id=c135afb53eaf43d9a99716160f0ffd25, scope.project.id=a9f733493ca84fbf8da49edbef53b518, user.id=e488658a09594272b20a5bf954380192) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:52.169011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d6bc3-bad3-408f-87be-0a9d0be0b375 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:52.170478 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25d6bc3-bad3-408f-87be-0a9d0be0b375 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:52.177736 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 150/294] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:51 2022] GET /identity/v3/role_assignments?user.id=e488658a09594272b20a5bf954380192&scope.project.id=a9f733493ca84fbf8da49edbef53b518&role.id=c135afb53eaf43d9a99716160f0ffd25 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:52.390418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cc485-d047-47af-85da-c7957b697594 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:52.390770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cc485-d047-47af-85da-c7957b697594 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:52.390965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0cc485-d047-47af-85da-c7957b697594 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:52.392564 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 145/295] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:52.398488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837c19d-cfb6-4b89-91e3-0bcf746ca4cf None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:52.398801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837c19d-cfb6-4b89-91e3-0bcf746ca4cf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:52.399041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837c19d-cfb6-4b89-91e3-0bcf746ca4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:52.418149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c837c19d-cfb6-4b89-91e3-0bcf746ca4cf None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:52.419282 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c837c19d-cfb6-4b89-91e3-0bcf746ca4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:52.506176 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 151/296] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:52.795193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cc485-d047-47af-85da-c7957b697594 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:52.916594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0cc485-d047-47af-85da-c7957b697594 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:52.952896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af1b95-9c54-4f04-af48-116bf6f03ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:52.953520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af1b95-9c54-4f04-af48-116bf6f03ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:52.953791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af1b95-9c54-4f04-af48-116bf6f03ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:52.954132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af1b95-9c54-4f04-af48-116bf6f03ac0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:52.964906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af1b95-9c54-4f04-af48-116bf6f03ac0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-f7af1b95-9c54-4f04-af48-116bf6f03ac0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:52.970574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:52.971496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:52.972302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:52.972302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:52.972302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:52.972302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:52.972302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:52.972302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:52.972302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:52.972302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:52.972302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:52.972302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:52.972302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:52.972302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:52.972302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 146/297] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:52.977568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837c19d-cfb6-4b89-91e3-0bcf746ca4cf None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:53.053034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:53.092415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837c19d-cfb6-4b89-91e3-0bcf746ca4cf None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:53.131749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c972648-1fba-4e1e-9fca-d7cd381b3a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:53.132356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c972648-1fba-4e1e-9fca-d7cd381b3a89 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:53.132630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c972648-1fba-4e1e-9fca-d7cd381b3a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:53.132898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c972648-1fba-4e1e-9fca-d7cd381b3a89 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:53.133553 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c972648-1fba-4e1e-9fca-d7cd381b3a89 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:53.133847 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c972648-1fba-4e1e-9fca-d7cd381b3a89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:53.135235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c972648-1fba-4e1e-9fca-d7cd381b3a89 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:53.143514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 152/298] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:53.149356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af1b95-9c54-4f04-af48-116bf6f03ac0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:53.166823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:53.202362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bc35b-6372-40fc-9545-9b95a8aae6bc None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:53.202679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bc35b-6372-40fc-9545-9b95a8aae6bc None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:53.202979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bc35b-6372-40fc-9545-9b95a8aae6bc None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:53.204374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 153/299] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:53.205238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e68b1e-e72d-4520-be05-7e3e11e97a6d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:53.205507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e68b1e-e72d-4520-be05-7e3e11e97a6d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:53.205775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e68b1e-e72d-4520-be05-7e3e11e97a6d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:53.207570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 154/300] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:53.210562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005aaae0-ccc2-4cc7-b427-a4fcd8c0d9cb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:53.211084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005aaae0-ccc2-4cc7-b427-a4fcd8c0d9cb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:53.211516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005aaae0-ccc2-4cc7-b427-a4fcd8c0d9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:53.230728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-005aaae0-ccc2-4cc7-b427-a4fcd8c0d9cb None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:53.231884 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-005aaae0-ccc2-4cc7-b427-a4fcd8c0d9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:53.257773 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af1b95-9c54-4f04-af48-116bf6f03ac0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:53.292850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e907688-c676-416c-badc-a4b4b84913b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:53.293469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e907688-c676-416c-badc-a4b4b84913b5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:53.293744 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e907688-c676-416c-badc-a4b4b84913b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:53.294013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e907688-c676-416c-badc-a4b4b84913b5 None admin] PATH_INFO: `/v3/users/e70d2494392a435093c29ce3caac6296` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:53.295236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e907688-c676-416c-badc-a4b4b84913b5 None admin] RBAC: Authorizing `identity:get_user(user_id=e70d2494392a435093c29ce3caac6296)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:53.296661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e907688-c676-416c-badc-a4b4b84913b5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:53.302910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 147/301] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:53 2022] GET /identity/v3/users/e70d2494392a435093c29ce3caac6296 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:53.303102 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 155/302] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:53.303188 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faaeab9-c0f4-423d-8fed-7969086d90cf None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:53.303188 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faaeab9-c0f4-423d-8fed-7969086d90cf None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:53.304676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005aaae0-ccc2-4cc7-b427-a4fcd8c0d9cb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:53.306007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faaeab9-c0f4-423d-8fed-7969086d90cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:53.329989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9faaeab9-c0f4-423d-8fed-7969086d90cf None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:53.331252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9faaeab9-c0f4-423d-8fed-7969086d90cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:53.392561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:53.403650 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 148/303] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:53.435899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005aaae0-ccc2-4cc7-b427-a4fcd8c0d9cb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:53.471166 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5488cec-5dde-48ba-b6d9-cedd0d56dc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:53.472131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5488cec-5dde-48ba-b6d9-cedd0d56dc1d None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:53.472641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5488cec-5dde-48ba-b6d9-cedd0d56dc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:53.473171 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5488cec-5dde-48ba-b6d9-cedd0d56dc1d None admin] PATH_INFO: `/v3/system/users/e70d2494392a435093c29ce3caac6296/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:53.474813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5488cec-5dde-48ba-b6d9-cedd0d56dc1d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e70d2494392a435093c29ce3caac6296, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:53.476394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5488cec-5dde-48ba-b6d9-cedd0d56dc1d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:53.488019 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 156/304] 10.222.0.55 () {58 vars in 1334 bytes} [Tue Sep 27 17:10:53 2022] PUT /identity/v3/system/users/e70d2494392a435093c29ce3caac6296/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:10:53.569512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faaeab9-c0f4-423d-8fed-7969086d90cf None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:53.582935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:53.679762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faaeab9-c0f4-423d-8fed-7969086d90cf None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:53.684361 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5488cec-5dde-48ba-b6d9-cedd0d56dc1d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:53.721680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afa90d-d84e-485a-b724-5ca336bcdd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:53.722615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afa90d-d84e-485a-b724-5ca336bcdd13 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:53.723463 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afa90d-d84e-485a-b724-5ca336bcdd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:53.723463 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afa90d-d84e-485a-b724-5ca336bcdd13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:53.724557 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84afa90d-d84e-485a-b724-5ca336bcdd13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:53.727856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84afa90d-d84e-485a-b724-5ca336bcdd13 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:53.731483 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 149/305] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:53.736432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afa90d-d84e-485a-b724-5ca336bcdd13 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:53.821557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5488cec-5dde-48ba-b6d9-cedd0d56dc1d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:53.833465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:53.855017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:53.861322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afa90d-d84e-485a-b724-5ca336bcdd13 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:53.868211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed4ae6-2b74-456e-81bf-c79281422c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:53.869274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed4ae6-2b74-456e-81bf-c79281422c2a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:53.869758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed4ae6-2b74-456e-81bf-c79281422c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:53.870228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed4ae6-2b74-456e-81bf-c79281422c2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:53.881436 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ed4ae6-2b74-456e-81bf-c79281422c2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-b2ed4ae6-2b74-456e-81bf-c79281422c2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:53.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:53.887692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:53.893631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:53.893631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:53.893631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:53.893631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:53.893631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:53.893631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:53.893631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:53.893631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:53.893631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:53.893631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:53.893631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:53.893631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:53.893631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 157/306] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:53.896218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed4ae6-2b74-456e-81bf-c79281422c2a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:53.916158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3d26e-d142-46e9-a27d-d784186ab49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:53.917389 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3d26e-d142-46e9-a27d-d784186ab49c None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:53.918451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3d26e-d142-46e9-a27d-d784186ab49c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:53.918725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3d26e-d142-46e9-a27d-d784186ab49c None admin] PATH_INFO: `/v3/groups` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:53.919371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3d26e-d142-46e9-a27d-d784186ab49c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:53.920799 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3d26e-d142-46e9-a27d-d784186ab49c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:53.940994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 150/307] 10.222.0.55 () {60 vars in 1134 bytes} [Tue Sep 27 17:10:53 2022] POST /identity/v3/groups => generated 229 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:10:53.940994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da5f5b-2173-4a3a-be41-0a106f2b24ac None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:53.940994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da5f5b-2173-4a3a-be41-0a106f2b24ac None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:53.940994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da5f5b-2173-4a3a-be41-0a106f2b24ac None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:53.940994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 151/308] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:53.949068 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519354d-e20c-4728-aa0a-fa0e635b909f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:53.949363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519354d-e20c-4728-aa0a-fa0e635b909f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:53.949611 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519354d-e20c-4728-aa0a-fa0e635b909f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:53.969134 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2519354d-e20c-4728-aa0a-fa0e635b909f None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:53.970433 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2519354d-e20c-4728-aa0a-fa0e635b909f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:54.023618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed4ae6-2b74-456e-81bf-c79281422c2a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:54.065953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 152/309] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:54.070744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4f3a7-5c76-4205-b4ef-cd65e34b8827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:54.071711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4f3a7-5c76-4205-b4ef-cd65e34b8827 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:54.072195 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4f3a7-5c76-4205-b4ef-cd65e34b8827 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:54.072677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4f3a7-5c76-4205-b4ef-cd65e34b8827 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:54.073623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4f3a7-5c76-4205-b4ef-cd65e34b8827 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:54.074165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4f3a7-5c76-4205-b4ef-cd65e34b8827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:54.075773 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4f3a7-5c76-4205-b4ef-cd65e34b8827 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:54.085790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 158/310] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:54.097663 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519354d-e20c-4728-aa0a-fa0e635b909f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:54.165127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:54.203719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519354d-e20c-4728-aa0a-fa0e635b909f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:54.239783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ab23a-4323-4171-930c-8c2ba0cb6a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:54.240415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ab23a-4323-4171-930c-8c2ba0cb6a2d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:54.240698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ab23a-4323-4171-930c-8c2ba0cb6a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:54.240966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ab23a-4323-4171-930c-8c2ba0cb6a2d None admin] PATH_INFO: `/v3/users/d81932eaadeb4a318bd9516f5c0caf81` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:54.255611 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ab23a-4323-4171-930c-8c2ba0cb6a2d None admin] RBAC: Authorizing `identity:get_user(user_id=d81932eaadeb4a318bd9516f5c0caf81)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:54.257081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ab23a-4323-4171-930c-8c2ba0cb6a2d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:54.259076 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 153/311] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:54 2022] GET /identity/v3/users/d81932eaadeb4a318bd9516f5c0caf81 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:54.264863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4f3a7-5c76-4205-b4ef-cd65e34b8827 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:54.317329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:54.351763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ab23a-4323-4171-930c-8c2ba0cb6a2d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:54.375056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4f3a7-5c76-4205-b4ef-cd65e34b8827 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:54.413789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebbcc6-5773-4220-95e6-89661c158d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:54.414671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebbcc6-5773-4220-95e6-89661c158d52 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:54.415186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebbcc6-5773-4220-95e6-89661c158d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:54.415753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebbcc6-5773-4220-95e6-89661c158d52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:54.416657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebbcc6-5773-4220-95e6-89661c158d52 None admin] RBAC: Adding query filter params (role.id=a002aab559bb446593ca855874db44a2, user.id=d81932eaadeb4a318bd9516f5c0caf81, scope.system=all) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:54.417097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebbcc6-5773-4220-95e6-89661c158d52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:54.418827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebbcc6-5773-4220-95e6-89661c158d52 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:54.430223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 159/312] 10.222.0.55 () {56 vars in 1308 bytes} [Tue Sep 27 17:10:54 2022] GET /identity/v3/role_assignments?user.id=d81932eaadeb4a318bd9516f5c0caf81&scope.system=all&role.id=a002aab559bb446593ca855874db44a2 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:54.479294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ab23a-4323-4171-930c-8c2ba0cb6a2d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:54.509568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:54.518781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65588aed-6122-407c-b03a-c45249f73d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:54.520400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65588aed-6122-407c-b03a-c45249f73d8f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:54.521021 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65588aed-6122-407c-b03a-c45249f73d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:54.521477 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65588aed-6122-407c-b03a-c45249f73d8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:54.533961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65588aed-6122-407c-b03a-c45249f73d8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-65588aed-6122-407c-b03a-c45249f73d8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:54.539632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:54.540562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:54.541768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:54.541768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:54.541768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:54.541768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:54.541768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:54.541768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:54.541768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:54.541768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:54.541768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:54.541768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:54.541768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:54.541768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:54.541768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 154/313] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:10:54.547252 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebbcc6-5773-4220-95e6-89661c158d52 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:54.657588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebbcc6-5773-4220-95e6-89661c158d52 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:54.671522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:54.694510 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7580a27-6d2b-4502-a26f-050159da9e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:54.695116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7580a27-6d2b-4502-a26f-050159da9e62 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:54.695395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7580a27-6d2b-4502-a26f-050159da9e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:54.695668 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7580a27-6d2b-4502-a26f-050159da9e62 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:54.696392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7580a27-6d2b-4502-a26f-050159da9e62 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:54.696699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7580a27-6d2b-4502-a26f-050159da9e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:54.698088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7580a27-6d2b-4502-a26f-050159da9e62 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:54.706632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 160/314] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:54.712515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65588aed-6122-407c-b03a-c45249f73d8f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:54.825609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65588aed-6122-407c-b03a-c45249f73d8f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:54.861837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25b954-9ab9-4d80-b7f5-b87386a007a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:54.862495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25b954-9ab9-4d80-b7f5-b87386a007a0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:54.862770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25b954-9ab9-4d80-b7f5-b87386a007a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:54.863037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25b954-9ab9-4d80-b7f5-b87386a007a0 None admin] PATH_INFO: `/v3/users/e1d2758336884ead9bb43c24b6f369f0` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:54.864220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25b954-9ab9-4d80-b7f5-b87386a007a0 None admin] RBAC: Authorizing `identity:get_user(user_id=e1d2758336884ead9bb43c24b6f369f0)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:54.865716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25b954-9ab9-4d80-b7f5-b87386a007a0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:54.867728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 155/315] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:54 2022] GET /identity/v3/users/e1d2758336884ead9bb43c24b6f369f0 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:54.872936 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7580a27-6d2b-4502-a26f-050159da9e62 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:54.984479 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7580a27-6d2b-4502-a26f-050159da9e62 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:55.022834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf49887-789f-4f4f-acc0-e14ff7db86f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:55.023779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf49887-789f-4f4f-acc0-e14ff7db86f2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:55.024717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf49887-789f-4f4f-acc0-e14ff7db86f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:55.025174 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf49887-789f-4f4f-acc0-e14ff7db86f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:55.083831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:55.118158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf49887-789f-4f4f-acc0-e14ff7db86f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:55.120700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf49887-789f-4f4f-acc0-e14ff7db86f2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:55.226086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 161/316] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:10:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:10:55.238169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25b954-9ab9-4d80-b7f5-b87386a007a0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:55.293352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:55.340242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25b954-9ab9-4d80-b7f5-b87386a007a0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:55.380925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f90025-8a8a-4a3c-9db9-e6788dd8e126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:55.381577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f90025-8a8a-4a3c-9db9-e6788dd8e126 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:55.381856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f90025-8a8a-4a3c-9db9-e6788dd8e126 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:55.382159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f90025-8a8a-4a3c-9db9-e6788dd8e126 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:55.383426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f90025-8a8a-4a3c-9db9-e6788dd8e126 None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:55.384887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f90025-8a8a-4a3c-9db9-e6788dd8e126 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:55.386931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 156/317] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:10:55 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:55.393148 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf49887-789f-4f4f-acc0-e14ff7db86f2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:55.485042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:55.512944 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf49887-789f-4f4f-acc0-e14ff7db86f2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:55.547314 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2005f6-abfb-45f3-9455-e6626f962717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:55.547911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2005f6-abfb-45f3-9455-e6626f962717 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:55.548184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2005f6-abfb-45f3-9455-e6626f962717 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:55.548449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2005f6-abfb-45f3-9455-e6626f962717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:55.549163 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2005f6-abfb-45f3-9455-e6626f962717 None admin] RBAC: Adding query filter params (role.id=a002aab559bb446593ca855874db44a2, scope.project.id=a9f733493ca84fbf8da49edbef53b518, user.id=e1d2758336884ead9bb43c24b6f369f0) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:55.549465 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2005f6-abfb-45f3-9455-e6626f962717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:55.551851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2005f6-abfb-45f3-9455-e6626f962717 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:55.558033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 162/318] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:10:55 2022] GET /identity/v3/role_assignments?user.id=e1d2758336884ead9bb43c24b6f369f0&scope.project.id=a9f733493ca84fbf8da49edbef53b518&role.id=a002aab559bb446593ca855874db44a2 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:55.791168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e77bc-553a-46bc-b24b-9849eae2ae74 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:55.791344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e77bc-553a-46bc-b24b-9849eae2ae74 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:55.791623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e77bc-553a-46bc-b24b-9849eae2ae74 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:55.793197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 157/319] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:55.798778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2097495-b879-4cc0-a1fe-1dd5de250ef5 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:55.799067 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2097495-b879-4cc0-a1fe-1dd5de250ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:55.799333 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2097495-b879-4cc0-a1fe-1dd5de250ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:55.819000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e2097495-b879-4cc0-a1fe-1dd5de250ef5 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:55.820198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e2097495-b879-4cc0-a1fe-1dd5de250ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:55.888026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 163/320] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:56.162238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e77bc-553a-46bc-b24b-9849eae2ae74 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:56.240177 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8d436-cd94-41ed-b142-dad44e2bdca3 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:56.240695 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8d436-cd94-41ed-b142-dad44e2bdca3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:56.240779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8d436-cd94-41ed-b142-dad44e2bdca3 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:56.242098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 164/321] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:56.247046 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a02ad-a792-44e7-94b4-8c7bb1d619d0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:56.247321 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a02ad-a792-44e7-94b4-8c7bb1d619d0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:56.247588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a02ad-a792-44e7-94b4-8c7bb1d619d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:56.266743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c27a02ad-a792-44e7-94b4-8c7bb1d619d0 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:56.267917 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c27a02ad-a792-44e7-94b4-8c7bb1d619d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:56.268123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e77bc-553a-46bc-b24b-9849eae2ae74 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:56.293015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:56.310232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21b440-edb2-46dd-8890-1505aaaaa7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:56.310866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21b440-edb2-46dd-8890-1505aaaaa7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:56.311138 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21b440-edb2-46dd-8890-1505aaaaa7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:56.311405 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e21b440-edb2-46dd-8890-1505aaaaa7f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:56.324395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e21b440-edb2-46dd-8890-1505aaaaa7f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7e21b440-edb2-46dd-8890-1505aaaaa7f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:56.329213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:56.330112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:56.330914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:56.330914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:56.330914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:56.330914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:56.330914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:56.330914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:56.330914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:56.330914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:56.330914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:56.330914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:56.330914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:10:56.330914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:10:56.330914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 158/322] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:56.337582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21b440-edb2-46dd-8890-1505aaaaa7f8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:56.345692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:56.361645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 165/323] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:56.442169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e21b440-edb2-46dd-8890-1505aaaaa7f8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:56.476093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6faaf-82d0-415a-a3ed-aaec04d6580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:56.476684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6faaf-82d0-415a-a3ed-aaec04d6580c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:56.476958 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6faaf-82d0-415a-a3ed-aaec04d6580c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:56.477224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e6faaf-82d0-415a-a3ed-aaec04d6580c None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:56.477875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6faaf-82d0-415a-a3ed-aaec04d6580c None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:56.478199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6faaf-82d0-415a-a3ed-aaec04d6580c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:56.479558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e6faaf-82d0-415a-a3ed-aaec04d6580c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:56.487920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 159/324] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:56.493986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a02ad-a792-44e7-94b4-8c7bb1d619d0 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:56.595908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a02ad-a792-44e7-94b4-8c7bb1d619d0 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:56.622044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6faaf-82d0-415a-a3ed-aaec04d6580c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:56.633686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b836b7-bb48-41a2-8eb7-303f21e3d27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:56.634348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b836b7-bb48-41a2-8eb7-303f21e3d27c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:56.634627 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b836b7-bb48-41a2-8eb7-303f21e3d27c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:56.634896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b836b7-bb48-41a2-8eb7-303f21e3d27c None admin] PATH_INFO: `/v3/users/e70d2494392a435093c29ce3caac6296` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:56.636012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b836b7-bb48-41a2-8eb7-303f21e3d27c None admin] RBAC: Authorizing `identity:get_user(user_id=e70d2494392a435093c29ce3caac6296)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:56.637461 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b836b7-bb48-41a2-8eb7-303f21e3d27c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:56.643923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 166/325] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:56 2022] GET /identity/v3/users/e70d2494392a435093c29ce3caac6296 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:56.644737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b836b7-bb48-41a2-8eb7-303f21e3d27c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:56.673940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:56.709541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:56.732294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e6faaf-82d0-415a-a3ed-aaec04d6580c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:56.744575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b836b7-bb48-41a2-8eb7-303f21e3d27c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:56.767103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f98dd-15c5-4070-80ae-42998e7b3b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:56.767722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f98dd-15c5-4070-80ae-42998e7b3b6b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:56.767997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f98dd-15c5-4070-80ae-42998e7b3b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:56.768264 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f98dd-15c5-4070-80ae-42998e7b3b6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:56.769360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f98dd-15c5-4070-80ae-42998e7b3b6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:56.770895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f98dd-15c5-4070-80ae-42998e7b3b6b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:56.772450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 160/326] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:10:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:56.775327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb4484-3bc1-4ea2-ac57-6a3d79ffccb6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:56.775610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb4484-3bc1-4ea2-ac57-6a3d79ffccb6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:56.775887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb4484-3bc1-4ea2-ac57-6a3d79ffccb6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:56.777095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1430c1f-c903-4cc6-a538-248fa615881d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:56.777321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 161/327] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:56.777693 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1430c1f-c903-4cc6-a538-248fa615881d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:56.777797 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb4484-3bc1-4ea2-ac57-6a3d79ffccb6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:56.777975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1430c1f-c903-4cc6-a538-248fa615881d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:56.778262 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1430c1f-c903-4cc6-a538-248fa615881d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:56.778981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1430c1f-c903-4cc6-a538-248fa615881d None admin] RBAC: Adding query filter params (role.id=c135afb53eaf43d9a99716160f0ffd25, user.id=e70d2494392a435093c29ce3caac6296, scope.system=all) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:56.779283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1430c1f-c903-4cc6-a538-248fa615881d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:56.780695 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1430c1f-c903-4cc6-a538-248fa615881d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:56.792507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 167/328] 10.222.0.55 () {56 vars in 1308 bytes} [Tue Sep 27 17:10:56 2022] GET /identity/v3/role_assignments?user.id=e70d2494392a435093c29ce3caac6296&scope.system=all&role.id=c135afb53eaf43d9a99716160f0ffd25 => generated 510 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:10:56.795675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cb8a9-5a33-4b6b-bdd9-f019f8df881b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:56.796108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cb8a9-5a33-4b6b-bdd9-f019f8df881b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:56.796388 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cb8a9-5a33-4b6b-bdd9-f019f8df881b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:56.817916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c51cb8a9-5a33-4b6b-bdd9-f019f8df881b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:56.819426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c51cb8a9-5a33-4b6b-bdd9-f019f8df881b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:56.894667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 168/329] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:56.895579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb4484-3bc1-4ea2-ac57-6a3d79ffccb6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:56.909989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:56.934966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a69d3-584f-4595-ae4f-ff5606aa85f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:56.935739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a69d3-584f-4595-ae4f-ff5606aa85f1 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:56.936123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a69d3-584f-4595-ae4f-ff5606aa85f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:56.936569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a69d3-584f-4595-ae4f-ff5606aa85f1 None admin] PATH_INFO: `/v3/groups` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:56.937228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a69d3-584f-4595-ae4f-ff5606aa85f1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:56.938802 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a69d3-584f-4595-ae4f-ff5606aa85f1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:56.956559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 162/330] 10.222.0.55 () {60 vars in 1134 bytes} [Tue Sep 27 17:10:56 2022] POST /identity/v3/groups => generated 226 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:10:57.169064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cb8a9-5a33-4b6b-bdd9-f019f8df881b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:57.229537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:57.275713 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cb8a9-5a33-4b6b-bdd9-f019f8df881b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:57.317596 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d18ce-08ee-4ba9-9bc2-d598e8e90dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:57.318277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d18ce-08ee-4ba9-9bc2-d598e8e90dfe None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:57.318558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d18ce-08ee-4ba9-9bc2-d598e8e90dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:57.318830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d18ce-08ee-4ba9-9bc2-d598e8e90dfe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:57.329628 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d18ce-08ee-4ba9-9bc2-d598e8e90dfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-0c6d18ce-08ee-4ba9-9bc2-d598e8e90dfe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:10:57.335059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:10:57.335919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:10:57.336707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:10:57.336707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:10:57.336707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:10:57.336707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:10:57.336707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:10:57.336707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:10:57.336707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:10:57.336707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:10:57.336707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:10:57.336707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:10:57.336707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:10:57.336707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:10:57.338161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 169/331] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:10:57.342304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a69d3-584f-4595-ae4f-ff5606aa85f1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:57.451402 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a69d3-584f-4595-ae4f-ff5606aa85f1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:57.486367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c7a36-afd9-42af-ac68-7e70e2c719dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:57.487259 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c7a36-afd9-42af-ac68-7e70e2c719dc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:57.487712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c7a36-afd9-42af-ac68-7e70e2c719dc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:57.488666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c7a36-afd9-42af-ac68-7e70e2c719dc None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:57.489509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c7a36-afd9-42af-ac68-7e70e2c719dc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:10:57.490006 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c7a36-afd9-42af-ac68-7e70e2c719dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:57.491565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c7a36-afd9-42af-ac68-7e70e2c719dc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:57.500445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 163/332] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:10:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:57.506983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d18ce-08ee-4ba9-9bc2-d598e8e90dfe None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:57.623789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d18ce-08ee-4ba9-9bc2-d598e8e90dfe None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:57.640157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:57.662514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6a4b2-839b-40dc-9fd6-871977d4682d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:57.663147 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6a4b2-839b-40dc-9fd6-871977d4682d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:57.663426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6a4b2-839b-40dc-9fd6-871977d4682d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:57.663698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6a4b2-839b-40dc-9fd6-871977d4682d None admin] PATH_INFO: `/v3/users/d81932eaadeb4a318bd9516f5c0caf81` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:57.665140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6a4b2-839b-40dc-9fd6-871977d4682d None admin] RBAC: Authorizing `identity:get_user(user_id=d81932eaadeb4a318bd9516f5c0caf81)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:57.666687 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6a4b2-839b-40dc-9fd6-871977d4682d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:57.668430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 170/333] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:10:57 2022] GET /identity/v3/users/d81932eaadeb4a318bd9516f5c0caf81 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:10:57.674143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c7a36-afd9-42af-ac68-7e70e2c719dc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:57.719925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:57.803935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c7a36-afd9-42af-ac68-7e70e2c719dc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:10:57.842669 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e920c-2fef-4d89-b62f-0d60f384d786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:10:57.843325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e920c-2fef-4d89-b62f-0d60f384d786 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:57.843601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e920c-2fef-4d89-b62f-0d60f384d786 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:57.843871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e920c-2fef-4d89-b62f-0d60f384d786 None admin] PATH_INFO: `/v3/system/users/d81932eaadeb4a318bd9516f5c0caf81/roles/a002aab559bb446593ca855874db44a2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:57.845306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e920c-2fef-4d89-b62f-0d60f384d786 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d81932eaadeb4a318bd9516f5c0caf81, role_id=a002aab559bb446593ca855874db44a2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:10:57.846728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e920c-2fef-4d89-b62f-0d60f384d786 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:10:57.867441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 164/334] 10.222.0.55 () {58 vars in 1334 bytes} [Tue Sep 27 17:10:57 2022] PUT /identity/v3/system/users/d81932eaadeb4a318bd9516f5c0caf81/roles/a002aab559bb446593ca855874db44a2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:10:59.412620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0acda3-1008-41ea-9da1-0122ff80b3ab None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:59.413581 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0acda3-1008-41ea-9da1-0122ff80b3ab None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:59.414053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0acda3-1008-41ea-9da1-0122ff80b3ab None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:59.415884 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 171/335] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:59.421484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deead508-a59c-43bf-bd89-fa7c8d63be45 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:59.422385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deead508-a59c-43bf-bd89-fa7c8d63be45 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:59.422892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deead508-a59c-43bf-bd89-fa7c8d63be45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:59.444100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-deead508-a59c-43bf-bd89-fa7c8d63be45 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:59.445671 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-deead508-a59c-43bf-bd89-fa7c8d63be45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:59.529998 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 165/336] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:10:59.684928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ba4b-c1c8-4197-8b11-bdafb1715f9c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:59.685275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ba4b-c1c8-4197-8b11-bdafb1715f9c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:59.685458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861ba4b-c1c8-4197-8b11-bdafb1715f9c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:59.687031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 172/337] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:10:59.692435 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2079fb5-cd9d-4b8e-9a10-35492517600b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:10:59.692716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2079fb5-cd9d-4b8e-9a10-35492517600b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:10:59.692982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2079fb5-cd9d-4b8e-9a10-35492517600b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:10:59.718879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a2079fb5-cd9d-4b8e-9a10-35492517600b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:10:59.720058 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a2079fb5-cd9d-4b8e-9a10-35492517600b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:10:59.747756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:10:59.797550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 166/338] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:10:59.833416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861ba4b-c1c8-4197-8b11-bdafb1715f9c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:10:59.959654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861ba4b-c1c8-4197-8b11-bdafb1715f9c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:00.007543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270c0ea-c77a-431e-85a1-23dcd525bbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:00.008591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270c0ea-c77a-431e-85a1-23dcd525bbd3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:00.009088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270c0ea-c77a-431e-85a1-23dcd525bbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:00.009718 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270c0ea-c77a-431e-85a1-23dcd525bbd3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:00.026923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270c0ea-c77a-431e-85a1-23dcd525bbd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a270c0ea-c77a-431e-85a1-23dcd525bbd3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:00.032640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:00.033515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:00.034356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:00.034356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:00.034356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:00.034356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:00.034356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:00.034356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:00.034356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:00.034356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:00.034356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:00.034356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:00.034356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:11:00.034356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:00.036616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 173/339] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:10:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:00.042600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2079fb5-cd9d-4b8e-9a10-35492517600b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:00.187925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2079fb5-cd9d-4b8e-9a10-35492517600b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:00.227058 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71782a4-99d4-40b6-a770-798dcaf3633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:00.227573 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71782a4-99d4-40b6-a770-798dcaf3633e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:00.227844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71782a4-99d4-40b6-a770-798dcaf3633e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:00.228108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71782a4-99d4-40b6-a770-798dcaf3633e None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:00.228768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71782a4-99d4-40b6-a770-798dcaf3633e None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:00.229064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71782a4-99d4-40b6-a770-798dcaf3633e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:00.230488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71782a4-99d4-40b6-a770-798dcaf3633e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:00.238949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270c0ea-c77a-431e-85a1-23dcd525bbd3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:00.241858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 167/340] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:11:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:00.247333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71782a4-99d4-40b6-a770-798dcaf3633e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:00.272267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:00.322528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:00.380944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71782a4-99d4-40b6-a770-798dcaf3633e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:00.418431 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea2963-105f-4024-837e-c7ef63e33590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:00.420787 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea2963-105f-4024-837e-c7ef63e33590 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:00.421203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea2963-105f-4024-837e-c7ef63e33590 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:00.421608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea2963-105f-4024-837e-c7ef63e33590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:00.423921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270c0ea-c77a-431e-85a1-23dcd525bbd3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:00.492654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3893606-3a82-4690-93b8-30e73455b06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:00.493301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3893606-3a82-4690-93b8-30e73455b06e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:00.493584 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3893606-3a82-4690-93b8-30e73455b06e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:00.493853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3893606-3a82-4690-93b8-30e73455b06e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:00.503789 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea2963-105f-4024-837e-c7ef63e33590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:00.505478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea2963-105f-4024-837e-c7ef63e33590 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:00.513547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3893606-3a82-4690-93b8-30e73455b06e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-d3893606-3a82-4690-93b8-30e73455b06e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:00.525042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:00.526179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:00.527197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:00.527197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:00.527197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:00.527197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:00.527197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:00.527197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:00.527197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:00.527197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:00.527197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:00.527197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:00.527197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:00.527197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:00.529236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:00.535743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 174/341] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:11:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:00.541278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3893606-3a82-4690-93b8-30e73455b06e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:00.611280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 168/342] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 27 17:11:00.617866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea2963-105f-4024-837e-c7ef63e33590 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:00.708406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3893606-3a82-4690-93b8-30e73455b06e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:00.710215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:00.747525 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea2963-105f-4024-837e-c7ef63e33590 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:00.749493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143b01a-8668-4aaa-b11d-be5688dd96f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:00.750972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b01a-8668-4aaa-b11d-be5688dd96f5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:00.751269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b01a-8668-4aaa-b11d-be5688dd96f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:00.751539 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143b01a-8668-4aaa-b11d-be5688dd96f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:00.752271 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b01a-8668-4aaa-b11d-be5688dd96f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:00.752569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b01a-8668-4aaa-b11d-be5688dd96f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:00.755166 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143b01a-8668-4aaa-b11d-be5688dd96f5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:00.768112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 175/343] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:00.770844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723b68b-6e99-4e44-823d-cfad30f7580b None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:00.771343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723b68b-6e99-4e44-823d-cfad30f7580b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:00.771839 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2723b68b-6e99-4e44-823d-cfad30f7580b None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:00.773586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 176/344] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:00.774529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723b68b-6e99-4e44-823d-cfad30f7580b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:00.783074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:00.812183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909bacf-679f-40de-b36a-2cc670f815a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:00.812815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909bacf-679f-40de-b36a-2cc670f815a9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:00.813091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909bacf-679f-40de-b36a-2cc670f815a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:00.813363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909bacf-679f-40de-b36a-2cc670f815a9 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:00.814651 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909bacf-679f-40de-b36a-2cc670f815a9 None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:00.816616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909bacf-679f-40de-b36a-2cc670f815a9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:00.820508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 169/345] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:00 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:00.821775 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115458d5-e1dc-43d6-832b-b8e40e954e54 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:00.822051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115458d5-e1dc-43d6-832b-b8e40e954e54 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:00.827256 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115458d5-e1dc-43d6-832b-b8e40e954e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:00.863242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-115458d5-e1dc-43d6-832b-b8e40e954e54 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:00.864534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-115458d5-e1dc-43d6-832b-b8e40e954e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:00.984164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2723b68b-6e99-4e44-823d-cfad30f7580b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:01.010961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 170/346] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:01.011764 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115458d5-e1dc-43d6-832b-b8e40e954e54 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:01.063287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655f86f-a848-4e30-a858-21b30a0ff374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:01.064161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655f86f-a848-4e30-a858-21b30a0ff374 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:01.064579 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655f86f-a848-4e30-a858-21b30a0ff374 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:01.064987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655f86f-a848-4e30-a858-21b30a0ff374 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:01.124502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:01.185257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115458d5-e1dc-43d6-832b-b8e40e954e54 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:01.190587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655f86f-a848-4e30-a858-21b30a0ff374 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:01.193012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655f86f-a848-4e30-a858-21b30a0ff374 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:01.217104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:01.252808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19b07b-83c1-49f1-b1ed-cbc3ad01b92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:01.252808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19b07b-83c1-49f1-b1ed-cbc3ad01b92e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:01.252808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19b07b-83c1-49f1-b1ed-cbc3ad01b92e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:01.252808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19b07b-83c1-49f1-b1ed-cbc3ad01b92e None admin] PATH_INFO: `/v3/groups/feb35d9f372141cf9eb91fa375ed6239` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:01.261377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19b07b-83c1-49f1-b1ed-cbc3ad01b92e None admin] RBAC: Authorizing `identity:get_group(group_id=feb35d9f372141cf9eb91fa375ed6239)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:01.262811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19b07b-83c1-49f1-b1ed-cbc3ad01b92e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:01.264610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 171/347] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:01 2022] GET /identity/v3/groups/feb35d9f372141cf9eb91fa375ed6239 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:01.271708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19b07b-83c1-49f1-b1ed-cbc3ad01b92e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:01.370669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 177/348] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:01.377809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655f86f-a848-4e30-a858-21b30a0ff374 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:01.449154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19b07b-83c1-49f1-b1ed-cbc3ad01b92e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:01.472890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:01.516609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655f86f-a848-4e30-a858-21b30a0ff374 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:01.553636 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde40735-e520-4d88-9293-11109e96c9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:01.554663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde40735-e520-4d88-9293-11109e96c9ed None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:01.555132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde40735-e520-4d88-9293-11109e96c9ed None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:01.555589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde40735-e520-4d88-9293-11109e96c9ed None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:01.556893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde40735-e520-4d88-9293-11109e96c9ed None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:01.558574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde40735-e520-4d88-9293-11109e96c9ed None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:01.559688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581becb-90ed-46a7-b4e6-ff505af6104b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:01.560311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581becb-90ed-46a7-b4e6-ff505af6104b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:01.560589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581becb-90ed-46a7-b4e6-ff505af6104b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:01.560866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581becb-90ed-46a7-b4e6-ff505af6104b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:01.561603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581becb-90ed-46a7-b4e6-ff505af6104b None admin] RBAC: Adding query filter params (group.id=feb35d9f372141cf9eb91fa375ed6239, role.id=c135afb53eaf43d9a99716160f0ffd25, scope.project.id=00cafd6359074bd5ae371a50effe7297) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:01.561799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 178/349] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:01 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:01.561893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581becb-90ed-46a7-b4e6-ff505af6104b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:01.562830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde40735-e520-4d88-9293-11109e96c9ed None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:01.565186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581becb-90ed-46a7-b4e6-ff505af6104b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:01.585565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 172/350] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Sep 27 17:11:01 2022] GET /identity/v3/role_assignments?group.id=feb35d9f372141cf9eb91fa375ed6239&scope.project.id=00cafd6359074bd5ae371a50effe7297&role.id=c135afb53eaf43d9a99716160f0ffd25 => generated 260 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:01.587141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581becb-90ed-46a7-b4e6-ff505af6104b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:01.696089 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde40735-e520-4d88-9293-11109e96c9ed None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:01.733096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:01.738409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:01.741793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3517e98-9a28-4934-b2b4-af43ea5894fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:01.742591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3517e98-9a28-4934-b2b4-af43ea5894fe None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:01.742938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3517e98-9a28-4934-b2b4-af43ea5894fe None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:01.743213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3517e98-9a28-4934-b2b4-af43ea5894fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:01.756746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3517e98-9a28-4934-b2b4-af43ea5894fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-c3517e98-9a28-4934-b2b4-af43ea5894fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:01.762677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:01.763638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:01.764905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:01.764905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:01.764905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:01.764905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:01.764905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:01.764905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:01.764905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:01.764905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:01.764905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:01.764905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:01.764905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:11:01.764905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:01.764905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 179/351] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:01.765590 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d8f6f-22df-436c-802b-2f19fd6e37bd None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:01.765682 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d8f6f-22df-436c-802b-2f19fd6e37bd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:01.766120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d8f6f-22df-436c-802b-2f19fd6e37bd None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:01.768197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 180/352] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:01.774366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69964f6-d576-4570-87ce-811ca03735b8 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:01.775023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69964f6-d576-4570-87ce-811ca03735b8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:01.775434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69964f6-d576-4570-87ce-811ca03735b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:01.798571 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c69964f6-d576-4570-87ce-811ca03735b8 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:01.799996 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c69964f6-d576-4570-87ce-811ca03735b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:01.855635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581becb-90ed-46a7-b4e6-ff505af6104b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:01.883448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 181/353] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:01.884638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69964f6-d576-4570-87ce-811ca03735b8 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:01.891790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea242d-92c2-43af-bd0a-9f90ba656150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:01.892781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea242d-92c2-43af-bd0a-9f90ba656150 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:01.893204 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea242d-92c2-43af-bd0a-9f90ba656150 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:01.893647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea242d-92c2-43af-bd0a-9f90ba656150 None admin] PATH_INFO: `/v3/groups/feb35d9f372141cf9eb91fa375ed6239` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:01.895482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea242d-92c2-43af-bd0a-9f90ba656150 None admin] RBAC: Authorizing `identity:get_group(group_id=feb35d9f372141cf9eb91fa375ed6239)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:01.897052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea242d-92c2-43af-bd0a-9f90ba656150 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:01.898800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 173/354] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:01 2022] GET /identity/v3/groups/feb35d9f372141cf9eb91fa375ed6239 => generated 226 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:01.900147 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64538774-fd54-4bf8-9f25-e4de424667f9 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:01.900588 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64538774-fd54-4bf8-9f25-e4de424667f9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:01.901001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64538774-fd54-4bf8-9f25-e4de424667f9 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:01.902614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 174/355] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:01.907163 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64538774-fd54-4bf8-9f25-e4de424667f9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:01.992040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69964f6-d576-4570-87ce-811ca03735b8 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:02.015797 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64538774-fd54-4bf8-9f25-e4de424667f9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:02.016010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:02.032529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008913a9-f6fb-40aa-a483-7ed3f0c25a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:02.033356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008913a9-f6fb-40aa-a483-7ed3f0c25a10 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:02.033772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008913a9-f6fb-40aa-a483-7ed3f0c25a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:02.034237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008913a9-f6fb-40aa-a483-7ed3f0c25a10 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:02.035151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008913a9-f6fb-40aa-a483-7ed3f0c25a10 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:02.035673 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008913a9-f6fb-40aa-a483-7ed3f0c25a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:02.037215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008913a9-f6fb-40aa-a483-7ed3f0c25a10 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:02.044035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:02.048051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 182/356] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:11:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:02.049495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3e2a9-a36e-4f86-815b-bd3890919ae0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:02.049914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3e2a9-a36e-4f86-815b-bd3890919ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:02.050416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3e2a9-a36e-4f86-815b-bd3890919ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:02.060438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cf0eb-edc9-4390-ac91-ddf95a39bc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:02.061145 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf0eb-edc9-4390-ac91-ddf95a39bc5c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:02.061498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf0eb-edc9-4390-ac91-ddf95a39bc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:02.061841 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf0eb-edc9-4390-ac91-ddf95a39bc5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:02.062677 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cf0eb-edc9-4390-ac91-ddf95a39bc5c None admin] RBAC: Adding query filter params (group.id=feb35d9f372141cf9eb91fa375ed6239, role.id=dfb71501dfed4ce48cf8cb3559cee9d6, scope.project.id=00cafd6359074bd5ae371a50effe7297) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:02.063063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cf0eb-edc9-4390-ac91-ddf95a39bc5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:02.064567 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cf0eb-edc9-4390-ac91-ddf95a39bc5c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:02.072107 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-cbc3e2a9-a36e-4f86-815b-bd3890919ae0 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:02.072935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 175/357] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Sep 27 17:11:01 2022] GET /identity/v3/role_assignments?group.id=feb35d9f372141cf9eb91fa375ed6239&scope.project.id=00cafd6359074bd5ae371a50effe7297&role.id=dfb71501dfed4ce48cf8cb3559cee9d6 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:02.073639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cbc3e2a9-a36e-4f86-815b-bd3890919ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:02.073893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cf0eb-edc9-4390-ac91-ddf95a39bc5c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:02.157159 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 183/358] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:02.225179 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cf0eb-edc9-4390-ac91-ddf95a39bc5c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:02.267641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960785f6-3b7b-41a4-98e9-84be676d850e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:02.267923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960785f6-3b7b-41a4-98e9-84be676d850e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:02.268190 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960785f6-3b7b-41a4-98e9-84be676d850e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:02.269891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 184/359] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:11:02.272027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db0d9a-b3b8-455d-bac8-f48167b6721b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:02.272642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db0d9a-b3b8-455d-bac8-f48167b6721b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:02.272922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db0d9a-b3b8-455d-bac8-f48167b6721b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:02.273196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2db0d9a-b3b8-455d-bac8-f48167b6721b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:02.279554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fc32c-f4bd-47d3-bbc3-c6b4e23bb9d7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:02.279873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fc32c-f4bd-47d3-bbc3-c6b4e23bb9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:02.280095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fc32c-f4bd-47d3-bbc3-c6b4e23bb9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:02.306680 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5a6fc32c-f4bd-47d3-bbc3-c6b4e23bb9d7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:02.308345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5a6fc32c-f4bd-47d3-bbc3-c6b4e23bb9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:02.390475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:02.395053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 185/360] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:02.423945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db0d9a-b3b8-455d-bac8-f48167b6721b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:02.426065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2db0d9a-b3b8-455d-bac8-f48167b6721b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:02.545680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 176/361] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:02.548906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fc32c-f4bd-47d3-bbc3-c6b4e23bb9d7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:02.552196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db0d9a-b3b8-455d-bac8-f48167b6721b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:02.640671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:02.670496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:02.696955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2db0d9a-b3b8-455d-bac8-f48167b6721b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:02.728775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fc32c-f4bd-47d3-bbc3-c6b4e23bb9d7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:02.748581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f53257-e902-467d-98ec-5cc596c52608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:02.749215 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f53257-e902-467d-98ec-5cc596c52608 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:02.749682 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f53257-e902-467d-98ec-5cc596c52608 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:02.749788 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f53257-e902-467d-98ec-5cc596c52608 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:02.754770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f53257-e902-467d-98ec-5cc596c52608 None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:02.756219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f53257-e902-467d-98ec-5cc596c52608 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:02.758186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 177/362] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:02 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:02.758971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f53257-e902-467d-98ec-5cc596c52608 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:02.770936 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62468628-8bd9-49d2-a049-2f40958d10b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:02.771841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62468628-8bd9-49d2-a049-2f40958d10b0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:02.772316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62468628-8bd9-49d2-a049-2f40958d10b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:02.772759 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62468628-8bd9-49d2-a049-2f40958d10b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:02.784280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62468628-8bd9-49d2-a049-2f40958d10b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-62468628-8bd9-49d2-a049-2f40958d10b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:02.789501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:02.790468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:02.791283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:02.791283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:02.791283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:02.791283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:02.791283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:02.791283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:02.791283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:02.791283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:02.791283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:02.791283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:02.791283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:02.791283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:02.793604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 186/363] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:11:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:02.794684 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62468628-8bd9-49d2-a049-2f40958d10b0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:02.876920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f53257-e902-467d-98ec-5cc596c52608 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:02.904470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62468628-8bd9-49d2-a049-2f40958d10b0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:02.907196 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:02.912701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c693f6a-b3f5-4c30-b7e1-15d659c4e2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:02.913336 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c693f6a-b3f5-4c30-b7e1-15d659c4e2cc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:02.913616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c693f6a-b3f5-4c30-b7e1-15d659c4e2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:02.913884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c693f6a-b3f5-4c30-b7e1-15d659c4e2cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:02.932035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c693f6a-b3f5-4c30-b7e1-15d659c4e2cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-8c693f6a-b3f5-4c30-b7e1-15d659c4e2cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:02.937351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:02.938289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:02.939143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:02.939143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:02.939143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:02.939143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:02.939143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:02.939143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:02.939143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:02.939143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:02.939143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:02.939143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:02.939143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 27 17:11:02.939143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:02.939143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 178/364] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:02.943111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c693f6a-b3f5-4c30-b7e1-15d659c4e2cc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:02.948784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd28b5a-4638-4422-95b3-982452b9d904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:02.949633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd28b5a-4638-4422-95b3-982452b9d904 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:02.950051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd28b5a-4638-4422-95b3-982452b9d904 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:02.950484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd28b5a-4638-4422-95b3-982452b9d904 None admin] PATH_INFO: `/v3/groups/feb35d9f372141cf9eb91fa375ed6239` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:02.951758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd28b5a-4638-4422-95b3-982452b9d904 None admin] RBAC: Authorizing `identity:get_group(group_id=feb35d9f372141cf9eb91fa375ed6239)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:02.953339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd28b5a-4638-4422-95b3-982452b9d904 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:02.955385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 187/365] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:02 2022] GET /identity/v3/groups/feb35d9f372141cf9eb91fa375ed6239 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:02.956370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd28b5a-4638-4422-95b3-982452b9d904 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:02.958406 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:03.060472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c693f6a-b3f5-4c30-b7e1-15d659c4e2cc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:03.068357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd28b5a-4638-4422-95b3-982452b9d904 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:03.099280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b580c-d795-45dc-ac3c-c88446bfc84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:03.100022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b580c-d795-45dc-ac3c-c88446bfc84e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:03.100303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b580c-d795-45dc-ac3c-c88446bfc84e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:03.100572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b580c-d795-45dc-ac3c-c88446bfc84e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:03.108736 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f2d6b-f78b-4165-99d1-6ef8f4fd498b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:03.110156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f2d6b-f78b-4165-99d1-6ef8f4fd498b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:03.110649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f2d6b-f78b-4165-99d1-6ef8f4fd498b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:03.111114 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f2d6b-f78b-4165-99d1-6ef8f4fd498b None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:03.111982 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f2d6b-f78b-4165-99d1-6ef8f4fd498b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:03.112430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2b580c-d795-45dc-ac3c-c88446bfc84e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:03.112703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f2d6b-f78b-4165-99d1-6ef8f4fd498b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:03.114399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f2d6b-f78b-4165-99d1-6ef8f4fd498b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0e2b580c-d795-45dc-ac3c-c88446bfc84e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:03.117706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:03.118674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:03.119482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:03.119482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:03.119482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:03.119482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:03.119482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:03.119482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:03.119482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:03.119482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:03.119482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:03.119482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:03.119482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:03.119482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:03.119482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 179/366] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:03.120140 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b580c-d795-45dc-ac3c-c88446bfc84e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:03.126528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 188/367] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:03.127672 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f2d6b-f78b-4165-99d1-6ef8f4fd498b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:03.165890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:03.230522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:03.237435 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2b580c-d795-45dc-ac3c-c88446bfc84e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:03.237741 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f2d6b-f78b-4165-99d1-6ef8f4fd498b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:03.276673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20f0f4-9536-4bcf-bf6d-2fcd79d08a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:03.277266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20f0f4-9536-4bcf-bf6d-2fcd79d08a0e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:03.277540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20f0f4-9536-4bcf-bf6d-2fcd79d08a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:03.277848 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20f0f4-9536-4bcf-bf6d-2fcd79d08a0e None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:03.278528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20f0f4-9536-4bcf-bf6d-2fcd79d08a0e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:03.278812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20f0f4-9536-4bcf-bf6d-2fcd79d08a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:03.280365 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd054d0a-e4e3-45ef-b506-b1f9194680ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:03.280487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20f0f4-9536-4bcf-bf6d-2fcd79d08a0e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:03.281280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd054d0a-e4e3-45ef-b506-b1f9194680ed None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:03.281770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd054d0a-e4e3-45ef-b506-b1f9194680ed None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:03.282316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd054d0a-e4e3-45ef-b506-b1f9194680ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:03.283714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd054d0a-e4e3-45ef-b506-b1f9194680ed None admin] RBAC: Adding query filter params (group.id=feb35d9f372141cf9eb91fa375ed6239, role.id=c135afb53eaf43d9a99716160f0ffd25, scope.project.id=a9f733493ca84fbf8da49edbef53b518) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:03.284580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd054d0a-e4e3-45ef-b506-b1f9194680ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:03.287345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd054d0a-e4e3-45ef-b506-b1f9194680ed None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:03.290086 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 180/368] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:11:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:03.292044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20f0f4-9536-4bcf-bf6d-2fcd79d08a0e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:03.294534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 189/369] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Sep 27 17:11:03 2022] GET /identity/v3/role_assignments?group.id=feb35d9f372141cf9eb91fa375ed6239&scope.project.id=a9f733493ca84fbf8da49edbef53b518&role.id=c135afb53eaf43d9a99716160f0ffd25 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:03.295591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd054d0a-e4e3-45ef-b506-b1f9194680ed None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:03.411024 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:03.415073 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd054d0a-e4e3-45ef-b506-b1f9194680ed None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:03.455751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b99ca-e68e-4b07-906e-0771d64cc133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:03.457279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b99ca-e68e-4b07-906e-0771d64cc133 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:03.457766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b99ca-e68e-4b07-906e-0771d64cc133 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:03.458366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b99ca-e68e-4b07-906e-0771d64cc133 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:03.490383 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20f0f4-9536-4bcf-bf6d-2fcd79d08a0e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:03.532211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b99ca-e68e-4b07-906e-0771d64cc133 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:03.534279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b99ca-e68e-4b07-906e-0771d64cc133 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:03.538237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:03.580414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63de35-748f-4f62-9eca-5edc778fb865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:03.580414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63de35-748f-4f62-9eca-5edc778fb865 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:03.580414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63de35-748f-4f62-9eca-5edc778fb865 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:03.580414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63de35-748f-4f62-9eca-5edc778fb865 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:03.581282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63de35-748f-4f62-9eca-5edc778fb865 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:03.581282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63de35-748f-4f62-9eca-5edc778fb865 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:03.584236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63de35-748f-4f62-9eca-5edc778fb865 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:03.607057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 181/370] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:03.607757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63de35-748f-4f62-9eca-5edc778fb865 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:03.650337 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:03.653366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 190/371] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:03.655556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b99ca-e68e-4b07-906e-0771d64cc133 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:03.802809 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63de35-748f-4f62-9eca-5edc778fb865 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:03.840000 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a478e-46ca-4827-8a94-e84caa516e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:03.840793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a478e-46ca-4827-8a94-e84caa516e7a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:03.841076 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a478e-46ca-4827-8a94-e84caa516e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:03.841348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a478e-46ca-4827-8a94-e84caa516e7a None admin] PATH_INFO: `/v3/users/d81932eaadeb4a318bd9516f5c0caf81` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:03.842536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a478e-46ca-4827-8a94-e84caa516e7a None admin] RBAC: Authorizing `identity:get_user(user_id=d81932eaadeb4a318bd9516f5c0caf81)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:03.843219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b99ca-e68e-4b07-906e-0771d64cc133 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:03.843962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a478e-46ca-4827-8a94-e84caa516e7a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:03.845904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 182/372] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:11:03 2022] GET /identity/v3/users/d81932eaadeb4a318bd9516f5c0caf81 => generated 293 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:03.846709 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a478e-46ca-4827-8a94-e84caa516e7a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:03.918461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:03.942802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9143e-31bb-4e89-bca2-a2193121a59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:03.943425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9143e-31bb-4e89-bca2-a2193121a59c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:03.943701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9143e-31bb-4e89-bca2-a2193121a59c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:03.943974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9143e-31bb-4e89-bca2-a2193121a59c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:03.963451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a478e-46ca-4827-8a94-e84caa516e7a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:04.002119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f4598-c8e5-4315-b1df-ae22b0e1839c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:04.003119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f4598-c8e5-4315-b1df-ae22b0e1839c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:04.003443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f4598-c8e5-4315-b1df-ae22b0e1839c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:04.003820 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f4598-c8e5-4315-b1df-ae22b0e1839c None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:04.004971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f4598-c8e5-4315-b1df-ae22b0e1839c None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:04.006572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f4598-c8e5-4315-b1df-ae22b0e1839c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:04.008204 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 183/373] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:03 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:04.008977 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f4598-c8e5-4315-b1df-ae22b0e1839c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:04.070226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:04.076011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9143e-31bb-4e89-bca2-a2193121a59c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:04.077480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9143e-31bb-4e89-bca2-a2193121a59c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:04.129662 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f4598-c8e5-4315-b1df-ae22b0e1839c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:04.162505 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:04.168261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97caca85-06b8-48ab-8c3a-bcbf78316319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:04.169284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caca85-06b8-48ab-8c3a-bcbf78316319 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:04.169707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caca85-06b8-48ab-8c3a-bcbf78316319 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:04.170162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caca85-06b8-48ab-8c3a-bcbf78316319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:04.171047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97caca85-06b8-48ab-8c3a-bcbf78316319 None admin] RBAC: Adding query filter params (role.id=a002aab559bb446593ca855874db44a2, user.id=d81932eaadeb4a318bd9516f5c0caf81, scope.system=all) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:04.171516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97caca85-06b8-48ab-8c3a-bcbf78316319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:04.173320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97caca85-06b8-48ab-8c3a-bcbf78316319 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:04.192573 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 184/374] 10.222.0.55 () {56 vars in 1308 bytes} [Tue Sep 27 17:11:04 2022] GET /identity/v3/role_assignments?user.id=d81932eaadeb4a318bd9516f5c0caf81&scope.system=all&role.id=a002aab559bb446593ca855874db44a2 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:04.193772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97caca85-06b8-48ab-8c3a-bcbf78316319 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:04.246309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 191/375] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:04.252044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9143e-31bb-4e89-bca2-a2193121a59c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:04.352140 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97caca85-06b8-48ab-8c3a-bcbf78316319 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:04.399159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d308e3a-6f85-4c97-b5fc-753f14e5ef4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:04.400110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d308e3a-6f85-4c97-b5fc-753f14e5ef4a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:04.400544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d308e3a-6f85-4c97-b5fc-753f14e5ef4a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:04.400986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d308e3a-6f85-4c97-b5fc-753f14e5ef4a None admin] PATH_INFO: `/v3/groups/feb35d9f372141cf9eb91fa375ed6239` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:04.402613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d308e3a-6f85-4c97-b5fc-753f14e5ef4a None admin] RBAC: Authorizing `identity:get_group(group_id=feb35d9f372141cf9eb91fa375ed6239)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:04.404509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d308e3a-6f85-4c97-b5fc-753f14e5ef4a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:04.406850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 185/376] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:04 2022] GET /identity/v3/groups/feb35d9f372141cf9eb91fa375ed6239 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:04.414741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d308e3a-6f85-4c97-b5fc-753f14e5ef4a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:04.423834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:04.430311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9143e-31bb-4e89-bca2-a2193121a59c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:04.491571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:04.529291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a85fe-d787-4def-91f8-20c21505adf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:04.529941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a85fe-d787-4def-91f8-20c21505adf7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:04.530242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a85fe-d787-4def-91f8-20c21505adf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:04.530576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a85fe-d787-4def-91f8-20c21505adf7 None admin] PATH_INFO: `/v3/projects/5c93f41c1d2243bfbbfb547f2005e999` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:04.535379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a85fe-d787-4def-91f8-20c21505adf7 None admin] RBAC: Authorizing `identity:get_project(project_id=5c93f41c1d2243bfbbfb547f2005e999)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:04.536929 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a85fe-d787-4def-91f8-20c21505adf7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:04.538897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 192/377] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:04 2022] GET /identity/v3/projects/5c93f41c1d2243bfbbfb547f2005e999 => generated 344 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:04.546136 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a85fe-d787-4def-91f8-20c21505adf7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:04.546303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d308e3a-6f85-4c97-b5fc-753f14e5ef4a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:04.618436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa028b1-a26e-4744-9e3c-6588ebc41842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:04.619276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa028b1-a26e-4744-9e3c-6588ebc41842 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:04.619276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa028b1-a26e-4744-9e3c-6588ebc41842 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:04.619494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa028b1-a26e-4744-9e3c-6588ebc41842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:04.620192 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa028b1-a26e-4744-9e3c-6588ebc41842 None admin] RBAC: Adding query filter params (group.id=feb35d9f372141cf9eb91fa375ed6239, role.id=dfb71501dfed4ce48cf8cb3559cee9d6, scope.project.id=a9f733493ca84fbf8da49edbef53b518) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:04.620494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa028b1-a26e-4744-9e3c-6588ebc41842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:04.621931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa028b1-a26e-4744-9e3c-6588ebc41842 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:04.633498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 186/378] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Sep 27 17:11:04 2022] GET /identity/v3/role_assignments?group.id=feb35d9f372141cf9eb91fa375ed6239&scope.project.id=a9f733493ca84fbf8da49edbef53b518&role.id=dfb71501dfed4ce48cf8cb3559cee9d6 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:04.634922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4cb73-2c9d-420d-9f4f-50756070562e None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:04.635322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4cb73-2c9d-420d-9f4f-50756070562e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:04.635663 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4cb73-2c9d-420d-9f4f-50756070562e None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:04.637532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 187/379] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:04.644991 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7295aa9-3612-4ff0-81aa-5a349986181d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:04.645469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7295aa9-3612-4ff0-81aa-5a349986181d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:04.645925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7295aa9-3612-4ff0-81aa-5a349986181d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:04.675466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f7295aa9-3612-4ff0-81aa-5a349986181d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:04.677306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f7295aa9-3612-4ff0-81aa-5a349986181d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:04.721301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a85fe-d787-4def-91f8-20c21505adf7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:04.764594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 188/380] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:04.770263 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5be5f-90a6-49b6-9887-fb31792f67bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:04.771236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5be5f-90a6-49b6-9887-fb31792f67bb None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:04.771706 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5be5f-90a6-49b6-9887-fb31792f67bb None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:04.772164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5be5f-90a6-49b6-9887-fb31792f67bb None admin] PATH_INFO: `/v3/groups/7498960280744af982985dff55081bc3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:04.779558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5be5f-90a6-49b6-9887-fb31792f67bb None admin] RBAC: Authorizing `identity:get_group(group_id=7498960280744af982985dff55081bc3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:04.781332 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5be5f-90a6-49b6-9887-fb31792f67bb None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:04.783802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 193/381] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:04 2022] GET /identity/v3/groups/7498960280744af982985dff55081bc3 => generated 229 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:04.790532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7295aa9-3612-4ff0-81aa-5a349986181d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:04.862280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:04.924862 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7295aa9-3612-4ff0-81aa-5a349986181d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:04.975828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb1fded-3a93-48d6-b57c-0d7f0f2aae86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:04.976447 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1fded-3a93-48d6-b57c-0d7f0f2aae86 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:04.976725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1fded-3a93-48d6-b57c-0d7f0f2aae86 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:04.976995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb1fded-3a93-48d6-b57c-0d7f0f2aae86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:04.977797 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1fded-3a93-48d6-b57c-0d7f0f2aae86 None admin] RBAC: Adding query filter params (group.id=7498960280744af982985dff55081bc3, role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=5c93f41c1d2243bfbbfb547f2005e999) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:04.978050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1fded-3a93-48d6-b57c-0d7f0f2aae86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:04.979687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb1fded-3a93-48d6-b57c-0d7f0f2aae86 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:04.986911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 189/382] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Sep 27 17:11:04 2022] GET /identity/v3/role_assignments?group.id=7498960280744af982985dff55081bc3&scope.project.id=5c93f41c1d2243bfbbfb547f2005e999&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:05.092719 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5be5f-90a6-49b6-9887-fb31792f67bb None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:05.145338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99d396-a347-4f1d-8c1a-6752e62b8bc3 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:05.145776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99d396-a347-4f1d-8c1a-6752e62b8bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:05.146174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99d396-a347-4f1d-8c1a-6752e62b8bc3 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:05.146407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:05.147967 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 190/383] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:05.155153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f00fc-f218-4a80-ad15-086a15fd7732 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:05.155438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f00fc-f218-4a80-ad15-086a15fd7732 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:05.155593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f00fc-f218-4a80-ad15-086a15fd7732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:05.191857 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6c9f00fc-f218-4a80-ad15-086a15fd7732 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:05.193007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6c9f00fc-f218-4a80-ad15-086a15fd7732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:05.242020 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5be5f-90a6-49b6-9887-fb31792f67bb None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:05.310018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 191/384] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:11:05.310707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476abbb-fa14-45c8-9dda-f1afd60c0e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:05.311352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476abbb-fa14-45c8-9dda-f1afd60c0e6a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:05.311628 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476abbb-fa14-45c8-9dda-f1afd60c0e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:05.311898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476abbb-fa14-45c8-9dda-f1afd60c0e6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:05.332527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476abbb-fa14-45c8-9dda-f1afd60c0e6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-8476abbb-fa14-45c8-9dda-f1afd60c0e6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:05.339181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:05.340132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:05.340986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:05.340986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:05.340986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:05.340986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:05.340986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:05.340986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:05.340986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:05.340986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:05.340986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:05.340986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:05.340986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:11:05.340986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:05.343498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 194/385] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:05.350115 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f00fc-f218-4a80-ad15-086a15fd7732 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:05.402977 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:05.477103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f00fc-f218-4a80-ad15-086a15fd7732 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:05.517629 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044b68e-6361-4922-a470-406244f71db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:05.518281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044b68e-6361-4922-a470-406244f71db4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:05.518563 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044b68e-6361-4922-a470-406244f71db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:05.518836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d044b68e-6361-4922-a470-406244f71db4 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:05.520045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044b68e-6361-4922-a470-406244f71db4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:05.520346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044b68e-6361-4922-a470-406244f71db4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:05.521711 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d044b68e-6361-4922-a470-406244f71db4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:05.530468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 192/386] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:11:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:05.536380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476abbb-fa14-45c8-9dda-f1afd60c0e6a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:05.662426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:05.676416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476abbb-fa14-45c8-9dda-f1afd60c0e6a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:05.686464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044b68e-6361-4922-a470-406244f71db4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:05.720938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee8a17-a7f9-4f54-98c0-30a0ce43917b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:05.721578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee8a17-a7f9-4f54-98c0-30a0ce43917b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:05.722239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee8a17-a7f9-4f54-98c0-30a0ce43917b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:05.722239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee8a17-a7f9-4f54-98c0-30a0ce43917b None admin] PATH_INFO: `/v3/groups/feb35d9f372141cf9eb91fa375ed6239` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:05.723512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ee8a17-a7f9-4f54-98c0-30a0ce43917b None admin] RBAC: Authorizing `identity:get_group(group_id=feb35d9f372141cf9eb91fa375ed6239)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:05.725320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ee8a17-a7f9-4f54-98c0-30a0ce43917b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:05.728060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 195/387] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:05 2022] GET /identity/v3/groups/feb35d9f372141cf9eb91fa375ed6239 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:05.735056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee8a17-a7f9-4f54-98c0-30a0ce43917b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:05.869251 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d044b68e-6361-4922-a470-406244f71db4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:05.869251 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:05.908687 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee8a17-a7f9-4f54-98c0-30a0ce43917b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:05.909258 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82197dfd-d35d-4ded-af9d-958aa58345f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:05.910458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82197dfd-d35d-4ded-af9d-958aa58345f7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:05.910735 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82197dfd-d35d-4ded-af9d-958aa58345f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:05.911004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82197dfd-d35d-4ded-af9d-958aa58345f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:05.926096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82197dfd-d35d-4ded-af9d-958aa58345f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-82197dfd-d35d-4ded-af9d-958aa58345f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:05.927962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:05.928832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:05.930017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:05.930017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:05.930017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:05.930017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:05.930017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:05.930017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:05.930017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:05.930017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:05.930017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:05.930017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:05.930017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:05.930017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:05.930017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 193/388] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:11:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:05.934871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82197dfd-d35d-4ded-af9d-958aa58345f7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:05.983918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947a663-ab59-48ab-93df-81138454e7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:05.984927 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947a663-ab59-48ab-93df-81138454e7ec None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:05.985413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947a663-ab59-48ab-93df-81138454e7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:05.985903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947a663-ab59-48ab-93df-81138454e7ec None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:05.990834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947a663-ab59-48ab-93df-81138454e7ec None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:05.992594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947a663-ab59-48ab-93df-81138454e7ec None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:05.994954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 196/389] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:05 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:06.000346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947a663-ab59-48ab-93df-81138454e7ec None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:06.054969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82197dfd-d35d-4ded-af9d-958aa58345f7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:06.059627 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:06.099734 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f345e1-88e5-42fd-a507-5feddc02794e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:06.100766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f345e1-88e5-42fd-a507-5feddc02794e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:06.101325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f345e1-88e5-42fd-a507-5feddc02794e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:06.101737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f345e1-88e5-42fd-a507-5feddc02794e None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:06.102775 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f345e1-88e5-42fd-a507-5feddc02794e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:06.103237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f345e1-88e5-42fd-a507-5feddc02794e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:06.104851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f345e1-88e5-42fd-a507-5feddc02794e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:06.115501 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 194/390] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:06.121068 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f345e1-88e5-42fd-a507-5feddc02794e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:06.150420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:06.216484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947a663-ab59-48ab-93df-81138454e7ec None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:06.243764 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f345e1-88e5-42fd-a507-5feddc02794e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:06.281436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26744e0f-a438-4a0d-ae26-f1ac21b41bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:06.282461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26744e0f-a438-4a0d-ae26-f1ac21b41bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:06.282968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26744e0f-a438-4a0d-ae26-f1ac21b41bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:06.283433 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26744e0f-a438-4a0d-ae26-f1ac21b41bf5 None admin] PATH_INFO: `/v3/groups/feb35d9f372141cf9eb91fa375ed6239` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:06.284879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26744e0f-a438-4a0d-ae26-f1ac21b41bf5 None admin] RBAC: Authorizing `identity:get_group(group_id=feb35d9f372141cf9eb91fa375ed6239)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:06.286663 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26744e0f-a438-4a0d-ae26-f1ac21b41bf5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:06.288482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 195/391] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:06 2022] GET /identity/v3/groups/feb35d9f372141cf9eb91fa375ed6239 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:06.292972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d5b1e-f71b-420a-a2cc-f339e3723775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:06.292972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d5b1e-f71b-420a-a2cc-f339e3723775 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:06.292972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d5b1e-f71b-420a-a2cc-f339e3723775 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:06.292972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d5b1e-f71b-420a-a2cc-f339e3723775 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297/groups/feb35d9f372141cf9eb91fa375ed6239/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:06.294333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26744e0f-a438-4a0d-ae26-f1ac21b41bf5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:06.296939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d5b1e-f71b-420a-a2cc-f339e3723775 None admin] RBAC: Authorizing `identity:create_grant(project_id=00cafd6359074bd5ae371a50effe7297, group_id=feb35d9f372141cf9eb91fa375ed6239, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:06.298420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d5b1e-f71b-420a-a2cc-f339e3723775 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:06.324643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 197/392] 10.222.0.55 () {58 vars in 1442 bytes} [Tue Sep 27 17:11:05 2022] PUT /identity/v3/projects/00cafd6359074bd5ae371a50effe7297/groups/feb35d9f372141cf9eb91fa375ed6239/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:11:06.410102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:06.421642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26744e0f-a438-4a0d-ae26-f1ac21b41bf5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:06.461228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee821094-2679-46d6-b994-66557745139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:06.462291 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee821094-2679-46d6-b994-66557745139e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:06.462769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee821094-2679-46d6-b994-66557745139e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:06.463266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee821094-2679-46d6-b994-66557745139e None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:06.464704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee821094-2679-46d6-b994-66557745139e None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:06.466681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee821094-2679-46d6-b994-66557745139e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:06.469618 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 196/393] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:06 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:06.476327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d5b1e-f71b-420a-a2cc-f339e3723775 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:06.600639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:06.619738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d5b1e-f71b-420a-a2cc-f339e3723775 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:06.660435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f30387-5a22-4b87-af22-fda1143f1402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:06.661107 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f30387-5a22-4b87-af22-fda1143f1402 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:06.661391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f30387-5a22-4b87-af22-fda1143f1402 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:06.661660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f30387-5a22-4b87-af22-fda1143f1402 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297/groups/feb35d9f372141cf9eb91fa375ed6239/roles/dfb71501dfed4ce48cf8cb3559cee9d6` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:06.663903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f30387-5a22-4b87-af22-fda1143f1402 None admin] RBAC: Authorizing `identity:create_grant(project_id=00cafd6359074bd5ae371a50effe7297, group_id=feb35d9f372141cf9eb91fa375ed6239, role_id=dfb71501dfed4ce48cf8cb3559cee9d6)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:06.665346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f30387-5a22-4b87-af22-fda1143f1402 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:06.681743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 198/394] 10.222.0.55 () {58 vars in 1442 bytes} [Tue Sep 27 17:11:06 2022] PUT /identity/v3/projects/00cafd6359074bd5ae371a50effe7297/groups/feb35d9f372141cf9eb91fa375ed6239/roles/dfb71501dfed4ce48cf8cb3559cee9d6 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:11:06.947371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e308f1a-92da-43a9-bf82-5d319a40ed89 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:06.947861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e308f1a-92da-43a9-bf82-5d319a40ed89 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:06.947861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e308f1a-92da-43a9-bf82-5d319a40ed89 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:06.949407 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 197/395] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:06.958768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055692d7-e123-450e-981d-157324609cfa None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:06.959248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055692d7-e123-450e-981d-157324609cfa None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:06.959449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055692d7-e123-450e-981d-157324609cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:06.984291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-055692d7-e123-450e-981d-157324609cfa None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:06.985495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-055692d7-e123-450e-981d-157324609cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:07.073039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 199/396] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:07.387892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e308f1a-92da-43a9-bf82-5d319a40ed89 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:07.517975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e308f1a-92da-43a9-bf82-5d319a40ed89 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:07.557272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca6ddb-44a5-4a7d-803e-354a95d22dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:07.557901 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca6ddb-44a5-4a7d-803e-354a95d22dea None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:07.558209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca6ddb-44a5-4a7d-803e-354a95d22dea None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:07.558478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca6ddb-44a5-4a7d-803e-354a95d22dea None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:07.571919 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca6ddb-44a5-4a7d-803e-354a95d22dea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-25ca6ddb-44a5-4a7d-803e-354a95d22dea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:07.577282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:07.578225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:07.579034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:07.579034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:07.579034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:07.579034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:07.579034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:07.579034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:07.579034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:07.579034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:07.579034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:07.579034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:07.579034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:11:07.579034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:07.579034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 198/397] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:07.587202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055692d7-e123-450e-981d-157324609cfa None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:07.629769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21eba66-38ba-4fa1-980b-bc9a4a504fe1 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:07.630151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21eba66-38ba-4fa1-980b-bc9a4a504fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:07.630355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21eba66-38ba-4fa1-980b-bc9a4a504fe1 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:07.631941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 199/398] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:11:07.639030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685171b-03ce-4aac-b51e-210040eacca3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:07.639197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685171b-03ce-4aac-b51e-210040eacca3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:07.639436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685171b-03ce-4aac-b51e-210040eacca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:07.670888 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8685171b-03ce-4aac-b51e-210040eacca3 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:07.672097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8685171b-03ce-4aac-b51e-210040eacca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:07.702554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:07.721082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:07.764056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055692d7-e123-450e-981d-157324609cfa None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:07.801004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 200/399] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:07.816894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5662bc-3c3f-4978-9fcf-b23548ebaedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:07.817529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5662bc-3c3f-4978-9fcf-b23548ebaedd None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:07.817808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5662bc-3c3f-4978-9fcf-b23548ebaedd None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:07.818099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5662bc-3c3f-4978-9fcf-b23548ebaedd None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:07.818818 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5662bc-3c3f-4978-9fcf-b23548ebaedd None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:07.819125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5662bc-3c3f-4978-9fcf-b23548ebaedd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:07.820685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5662bc-3c3f-4978-9fcf-b23548ebaedd None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:07.830664 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 200/400] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:11:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:07.837194 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685171b-03ce-4aac-b51e-210040eacca3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:07.958941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3ef58-54cb-4568-b016-50e62e96ceff None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:07.959438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3ef58-54cb-4568-b016-50e62e96ceff None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:07.959512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3ef58-54cb-4568-b016-50e62e96ceff None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:07.961047 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 201/401] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:07.966690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef05f50-b795-4c91-8afd-c31eb0e862a9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:07.966917 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685171b-03ce-4aac-b51e-210040eacca3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:07.967303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef05f50-b795-4c91-8afd-c31eb0e862a9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:07.967303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef05f50-b795-4c91-8afd-c31eb0e862a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:08.003196 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1ef05f50-b795-4c91-8afd-c31eb0e862a9 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:08.004433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1ef05f50-b795-4c91-8afd-c31eb0e862a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:08.006654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02483ae9-7606-4f92-bdcc-c0d0eeb430d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:08.007394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02483ae9-7606-4f92-bdcc-c0d0eeb430d6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:08.007674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02483ae9-7606-4f92-bdcc-c0d0eeb430d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:08.007942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02483ae9-7606-4f92-bdcc-c0d0eeb430d6 None admin] PATH_INFO: `/v3/groups/feb35d9f372141cf9eb91fa375ed6239` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:08.009086 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02483ae9-7606-4f92-bdcc-c0d0eeb430d6 None admin] RBAC: Authorizing `identity:get_group(group_id=feb35d9f372141cf9eb91fa375ed6239)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:08.010591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02483ae9-7606-4f92-bdcc-c0d0eeb430d6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:08.012469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 201/402] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:07 2022] GET /identity/v3/groups/feb35d9f372141cf9eb91fa375ed6239 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:08.017876 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02483ae9-7606-4f92-bdcc-c0d0eeb430d6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:08.042414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:08.140958 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02483ae9-7606-4f92-bdcc-c0d0eeb430d6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:08.181059 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6726c-e019-4e09-9353-c58c7341654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:08.181698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6726c-e019-4e09-9353-c58c7341654b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:08.181982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6726c-e019-4e09-9353-c58c7341654b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:08.182297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6726c-e019-4e09-9353-c58c7341654b None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:08.183484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6726c-e019-4e09-9353-c58c7341654b None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:08.184932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6726c-e019-4e09-9353-c58c7341654b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:08.187586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 202/403] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:08 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:08.193674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6726c-e019-4e09-9353-c58c7341654b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:08.211408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 202/404] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:08.233408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef05f50-b795-4c91-8afd-c31eb0e862a9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:08.298467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:08.315568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6726c-e019-4e09-9353-c58c7341654b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:08.369542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:08.385333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22111e-055e-4643-9b2e-1005faf874cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:08.385999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22111e-055e-4643-9b2e-1005faf874cc None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:08.386307 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22111e-055e-4643-9b2e-1005faf874cc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:08.386581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22111e-055e-4643-9b2e-1005faf874cc None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518/groups/feb35d9f372141cf9eb91fa375ed6239/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:08.391817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22111e-055e-4643-9b2e-1005faf874cc None admin] RBAC: Authorizing `identity:create_grant(project_id=a9f733493ca84fbf8da49edbef53b518, group_id=feb35d9f372141cf9eb91fa375ed6239, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:08.393223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22111e-055e-4643-9b2e-1005faf874cc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:08.420253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 203/405] 10.222.0.55 () {58 vars in 1442 bytes} [Tue Sep 27 17:11:08 2022] PUT /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518/groups/feb35d9f372141cf9eb91fa375ed6239/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:11:08.420696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef05f50-b795-4c91-8afd-c31eb0e862a9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:08.459416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d5966-d572-4265-89bb-4e19af946c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:08.460067 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d5966-d572-4265-89bb-4e19af946c29 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:08.460345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d5966-d572-4265-89bb-4e19af946c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:08.460641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6d5966-d572-4265-89bb-4e19af946c29 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:08.473397 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6d5966-d572-4265-89bb-4e19af946c29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-9b6d5966-d572-4265-89bb-4e19af946c29 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:08.478703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:08.479593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:08.480404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:08.480404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:08.480404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:08.480404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:08.480404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:08.480404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:08.480404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:08.480404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:08.480404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:08.480404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:08.480404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:08.480404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:08.480404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 203/406] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:11:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:08.487237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22111e-055e-4643-9b2e-1005faf874cc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:08.560883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d5966-d572-4265-89bb-4e19af946c29 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:08.664224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22111e-055e-4643-9b2e-1005faf874cc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:08.672613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6d5966-d572-4265-89bb-4e19af946c29 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:08.705752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:08.711442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e64f3-38e9-48db-95bf-c4a52eb15e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:08.712052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e64f3-38e9-48db-95bf-c4a52eb15e51 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:08.712327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e64f3-38e9-48db-95bf-c4a52eb15e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:08.712534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab417a-c461-4999-8f59-a0fb86f31835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:08.712644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e64f3-38e9-48db-95bf-c4a52eb15e51 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:08.713249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e64f3-38e9-48db-95bf-c4a52eb15e51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:08.713360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab417a-c461-4999-8f59-a0fb86f31835 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:08.713548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e64f3-38e9-48db-95bf-c4a52eb15e51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:08.713644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab417a-c461-4999-8f59-a0fb86f31835 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:08.713934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab417a-c461-4999-8f59-a0fb86f31835 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:08.714935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e64f3-38e9-48db-95bf-c4a52eb15e51 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:08.725256 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 204/407] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:08.725526 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab417a-c461-4999-8f59-a0fb86f31835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:08.730673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e64f3-38e9-48db-95bf-c4a52eb15e51 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-5eab417a-c461-4999-8f59-a0fb86f31835 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:08.730792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:08.731907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:08.732690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:08.732690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:08.732690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:08.732690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:08.732690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:08.732690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:08.732690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:08.732690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:08.732690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:08.732690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:08.732690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:08.732690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:08.736737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 204/408] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:08.738771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab417a-c461-4999-8f59-a0fb86f31835 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:08.750022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:08.845978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab417a-c461-4999-8f59-a0fb86f31835 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:08.849067 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e64f3-38e9-48db-95bf-c4a52eb15e51 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:08.891845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40843fdb-f825-441f-b925-cca47b94f642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:08.892450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40843fdb-f825-441f-b925-cca47b94f642 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:08.892723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40843fdb-f825-441f-b925-cca47b94f642 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:08.892983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40843fdb-f825-441f-b925-cca47b94f642 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:08.893819 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40843fdb-f825-441f-b925-cca47b94f642 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:08.894642 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40843fdb-f825-441f-b925-cca47b94f642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:08.896012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40843fdb-f825-441f-b925-cca47b94f642 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:08.899638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb49b59-33f0-483c-a169-47feb5b87012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:08.900315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb49b59-33f0-483c-a169-47feb5b87012 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:08.900594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb49b59-33f0-483c-a169-47feb5b87012 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:08.900855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb49b59-33f0-483c-a169-47feb5b87012 None admin] PATH_INFO: `/v3/groups/feb35d9f372141cf9eb91fa375ed6239` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:08.903110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb49b59-33f0-483c-a169-47feb5b87012 None admin] RBAC: Authorizing `identity:get_group(group_id=feb35d9f372141cf9eb91fa375ed6239)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:08.904583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb49b59-33f0-483c-a169-47feb5b87012 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:08.906195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 205/409] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:08 2022] GET /identity/v3/groups/feb35d9f372141cf9eb91fa375ed6239 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:08.912422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb49b59-33f0-483c-a169-47feb5b87012 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:08.919846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 205/410] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:08.921087 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40843fdb-f825-441f-b925-cca47b94f642 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:09.038204 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:09.051395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:09.093664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb49b59-33f0-483c-a169-47feb5b87012 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:09.101430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40843fdb-f825-441f-b925-cca47b94f642 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:09.144746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4f9ab-b8dd-4e6c-ace6-f65906c5d395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:09.145483 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4f9ab-b8dd-4e6c-ace6-f65906c5d395 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:09.145762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4f9ab-b8dd-4e6c-ace6-f65906c5d395 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:09.146035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4f9ab-b8dd-4e6c-ace6-f65906c5d395 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:09.147218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4f9ab-b8dd-4e6c-ace6-f65906c5d395 None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:09.148712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4f9ab-b8dd-4e6c-ace6-f65906c5d395 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:09.150701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 206/411] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:08 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:09.151801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6b16-fa40-41d3-9c99-3b8dc6325829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:09.152441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6b16-fa40-41d3-9c99-3b8dc6325829 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:09.152716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6b16-fa40-41d3-9c99-3b8dc6325829 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:09.153366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6b16-fa40-41d3-9c99-3b8dc6325829 None admin] PATH_INFO: `/v3/groups/7498960280744af982985dff55081bc3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:09.155463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c6b16-fa40-41d3-9c99-3b8dc6325829 None admin] RBAC: Authorizing `identity:get_group(group_id=7498960280744af982985dff55081bc3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:09.156126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4f9ab-b8dd-4e6c-ace6-f65906c5d395 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:09.156882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c6b16-fa40-41d3-9c99-3b8dc6325829 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:09.158579 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 206/412] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:08 2022] GET /identity/v3/groups/7498960280744af982985dff55081bc3 => generated 229 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:09.166973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6b16-fa40-41d3-9c99-3b8dc6325829 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:09.267599 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4f9ab-b8dd-4e6c-ace6-f65906c5d395 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:09.305245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3011eac-8c15-4dce-925b-63490a221127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:09.306536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3011eac-8c15-4dce-925b-63490a221127 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:09.307016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3011eac-8c15-4dce-925b-63490a221127 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:09.307502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3011eac-8c15-4dce-925b-63490a221127 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518/groups/feb35d9f372141cf9eb91fa375ed6239/roles/dfb71501dfed4ce48cf8cb3559cee9d6` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:09.309937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3011eac-8c15-4dce-925b-63490a221127 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9f733493ca84fbf8da49edbef53b518, group_id=feb35d9f372141cf9eb91fa375ed6239, role_id=dfb71501dfed4ce48cf8cb3559cee9d6)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:09.312130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3011eac-8c15-4dce-925b-63490a221127 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:09.338516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 207/413] 10.222.0.55 () {58 vars in 1442 bytes} [Tue Sep 27 17:11:09 2022] PUT /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518/groups/feb35d9f372141cf9eb91fa375ed6239/roles/dfb71501dfed4ce48cf8cb3559cee9d6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:11:09.389052 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c6b16-fa40-41d3-9c99-3b8dc6325829 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:09.419846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:09.445840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa41b7a-6658-4fde-9a38-e3cdd4b05536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:09.446521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa41b7a-6658-4fde-9a38-e3cdd4b05536 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:09.446792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa41b7a-6658-4fde-9a38-e3cdd4b05536 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:09.447059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa41b7a-6658-4fde-9a38-e3cdd4b05536 None admin] PATH_INFO: `/v3/projects/5c93f41c1d2243bfbbfb547f2005e999` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:09.450932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa41b7a-6658-4fde-9a38-e3cdd4b05536 None admin] RBAC: Authorizing `identity:get_project(project_id=5c93f41c1d2243bfbbfb547f2005e999)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:09.452422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa41b7a-6658-4fde-9a38-e3cdd4b05536 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:09.454381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 207/414] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:09 2022] GET /identity/v3/projects/5c93f41c1d2243bfbbfb547f2005e999 => generated 344 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:09.460839 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3011eac-8c15-4dce-925b-63490a221127 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:09.465320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:09.494780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5a96f-3b49-4397-bbde-1371e45173fe None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:09.495115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5a96f-3b49-4397-bbde-1371e45173fe None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:09.495300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5a96f-3b49-4397-bbde-1371e45173fe None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:09.496895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 208/415] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:11:09.501985 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cea478-145c-4378-bea2-d19432f7bc2c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:09.502465 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cea478-145c-4378-bea2-d19432f7bc2c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:09.502747 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cea478-145c-4378-bea2-d19432f7bc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:09.525803 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c1cea478-145c-4378-bea2-d19432f7bc2c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:09.527017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c1cea478-145c-4378-bea2-d19432f7bc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:09.598158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3011eac-8c15-4dce-925b-63490a221127 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:09.626739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 209/416] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:11:09.630646 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b861f5-2d30-4982-a8d2-23d418f1349d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:09.631079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b861f5-2d30-4982-a8d2-23d418f1349d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:09.631495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b861f5-2d30-4982-a8d2-23d418f1349d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:09.633199 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 210/417] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:09.638494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd4792-cfc5-492f-a053-767bf41d8742 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:09.638914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd4792-cfc5-492f-a053-767bf41d8742 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:09.639284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd4792-cfc5-492f-a053-767bf41d8742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:09.658441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d8bd4792-cfc5-492f-a053-767bf41d8742 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:09.660354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d8bd4792-cfc5-492f-a053-767bf41d8742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:09.660580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffeaabe-5572-4589-9dc0-fd76edcda460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:09.661410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffeaabe-5572-4589-9dc0-fd76edcda460 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:09.661859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffeaabe-5572-4589-9dc0-fd76edcda460 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:09.662300 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffeaabe-5572-4589-9dc0-fd76edcda460 None admin] PATH_INFO: `/v3/projects/5c93f41c1d2243bfbbfb547f2005e999/groups/7498960280744af982985dff55081bc3/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:09.665992 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffeaabe-5572-4589-9dc0-fd76edcda460 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c93f41c1d2243bfbbfb547f2005e999, group_id=7498960280744af982985dff55081bc3, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:09.674106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffeaabe-5572-4589-9dc0-fd76edcda460 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:09.694949 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 208/418] 10.222.0.55 () {58 vars in 1442 bytes} [Tue Sep 27 17:11:09 2022] PUT /identity/v3/projects/5c93f41c1d2243bfbbfb547f2005e999/groups/7498960280744af982985dff55081bc3/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:11:09.760129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 211/419] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:11:09.920420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffeaabe-5572-4589-9dc0-fd76edcda460 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:09.990181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:10.031657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffeaabe-5572-4589-9dc0-fd76edcda460 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:10.066907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70161581-b403-474f-8a12-78328a5da9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:10.067548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161581-b403-474f-8a12-78328a5da9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:10.067777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161581-b403-474f-8a12-78328a5da9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:10.068045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161581-b403-474f-8a12-78328a5da9c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:10.072460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd4792-cfc5-492f-a053-767bf41d8742 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:10.083515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70161581-b403-474f-8a12-78328a5da9c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-70161581-b403-474f-8a12-78328a5da9c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:10.088806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:10.089687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:10.090520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:10.090520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:10.090520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:10.090520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:10.090520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:10.090520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:10.090520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:10.090520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:10.090520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:10.090520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:10.090520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:10.090520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:10.090520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 209/420] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:11:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:10.097621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70161581-b403-474f-8a12-78328a5da9c4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:10.161446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:10.203445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70161581-b403-474f-8a12-78328a5da9c4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:10.238638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:10.246920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e907f8-9103-4020-84e4-c00459b787c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:10.247534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e907f8-9103-4020-84e4-c00459b787c4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:10.247811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e907f8-9103-4020-84e4-c00459b787c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:10.248077 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e907f8-9103-4020-84e4-c00459b787c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:10.248745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e907f8-9103-4020-84e4-c00459b787c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:10.249331 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e907f8-9103-4020-84e4-c00459b787c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:10.250723 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e907f8-9103-4020-84e4-c00459b787c4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:10.266872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 210/421] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:10.270173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd4792-cfc5-492f-a053-767bf41d8742 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:10.274107 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e907f8-9103-4020-84e4-c00459b787c4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:10.356181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f205bd7-9131-4536-b8c3-a6343802dff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:10.357076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f205bd7-9131-4536-b8c3-a6343802dff5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:10.357356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f205bd7-9131-4536-b8c3-a6343802dff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:10.357632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f205bd7-9131-4536-b8c3-a6343802dff5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:10.378814 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e907f8-9103-4020-84e4-c00459b787c4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:10.399506 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f205bd7-9131-4536-b8c3-a6343802dff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-9f205bd7-9131-4536-b8c3-a6343802dff5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:10.404343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:10.405238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:10.406036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:10.406036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:10.406036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:10.406036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:10.406036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:10.406036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:10.406036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:10.406036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:10.406036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:10.406036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:10.406036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:11:10.406036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:10.410232 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 212/422] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:10.415896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f205bd7-9131-4536-b8c3-a6343802dff5 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:10.420743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90da54-9800-487b-a252-d32568ed24ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:10.421390 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90da54-9800-487b-a252-d32568ed24ad None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:10.421669 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90da54-9800-487b-a252-d32568ed24ad None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:10.421939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90da54-9800-487b-a252-d32568ed24ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:10.514188 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:10.531608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90da54-9800-487b-a252-d32568ed24ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:10.533041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90da54-9800-487b-a252-d32568ed24ad None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:10.597145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f205bd7-9131-4536-b8c3-a6343802dff5 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:10.614200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:10.686849 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65efb0-0d78-4bd8-abdc-db4595ca63ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:10.687471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65efb0-0d78-4bd8-abdc-db4595ca63ae None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:10.687744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65efb0-0d78-4bd8-abdc-db4595ca63ae None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:10.688006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65efb0-0d78-4bd8-abdc-db4595ca63ae None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:10.688678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65efb0-0d78-4bd8-abdc-db4595ca63ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:10.688976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65efb0-0d78-4bd8-abdc-db4595ca63ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:10.690400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65efb0-0d78-4bd8-abdc-db4595ca63ae None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:10.705169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 213/423] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:11:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:10.709200 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 211/424] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:10.711115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65efb0-0d78-4bd8-abdc-db4595ca63ae None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:10.715070 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90da54-9800-487b-a252-d32568ed24ad None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:10.864005 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:10.888767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65efb0-0d78-4bd8-abdc-db4595ca63ae None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:10.920233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90da54-9800-487b-a252-d32568ed24ad None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:10.929938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d664464-4460-4ed3-bc56-df92494532f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:10.931013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d664464-4460-4ed3-bc56-df92494532f1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:10.931487 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d664464-4460-4ed3-bc56-df92494532f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:10.931968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d664464-4460-4ed3-bc56-df92494532f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:10.992326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:10.993681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46148f0e-fe05-45a8-ad4b-454c46f034ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:10.994333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46148f0e-fe05-45a8-ad4b-454c46f034ef None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:10.995581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46148f0e-fe05-45a8-ad4b-454c46f034ef None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:10.995581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46148f0e-fe05-45a8-ad4b-454c46f034ef None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:11.000140 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46148f0e-fe05-45a8-ad4b-454c46f034ef None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:11.001606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46148f0e-fe05-45a8-ad4b-454c46f034ef None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:11.003356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 212/425] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:10 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:11.009721 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46148f0e-fe05-45a8-ad4b-454c46f034ef None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:11.062260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d664464-4460-4ed3-bc56-df92494532f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:11.063746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d664464-4460-4ed3-bc56-df92494532f1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:11.196823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46148f0e-fe05-45a8-ad4b-454c46f034ef None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:11.227699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 214/426] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:11.229665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c1891-6cb4-4eab-84e0-06d81efe4fc8 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:11.229945 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c1891-6cb4-4eab-84e0-06d81efe4fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:11.230240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c1891-6cb4-4eab-84e0-06d81efe4fc8 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:11.234145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 215/427] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:11.234909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c1891-6cb4-4eab-84e0-06d81efe4fc8 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:11.244161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba1d2d-35b2-4c60-94cb-d2330482c2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:11.245315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba1d2d-35b2-4c60-94cb-d2330482c2ff None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:11.245596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba1d2d-35b2-4c60-94cb-d2330482c2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:11.245865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba1d2d-35b2-4c60-94cb-d2330482c2ff None admin] PATH_INFO: `/v3/groups/feb35d9f372141cf9eb91fa375ed6239` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:11.247475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba1d2d-35b2-4c60-94cb-d2330482c2ff None admin] RBAC: Authorizing `identity:get_group(group_id=feb35d9f372141cf9eb91fa375ed6239)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:11.249625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba1d2d-35b2-4c60-94cb-d2330482c2ff None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:11.251569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 213/428] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:11 2022] GET /identity/v3/groups/feb35d9f372141cf9eb91fa375ed6239 => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:11.252681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d446c2-bad2-408b-b738-3b035eeaa528 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:11.253501 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d446c2-bad2-408b-b738-3b035eeaa528 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:11.253772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d446c2-bad2-408b-b738-3b035eeaa528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:11.276640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-87d446c2-bad2-408b-b738-3b035eeaa528 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:11.277837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-87d446c2-bad2-408b-b738-3b035eeaa528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:11.298331 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:11.314353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:11.359537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c1891-6cb4-4eab-84e0-06d81efe4fc8 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:11.366560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 214/429] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:11.367511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d446c2-bad2-408b-b738-3b035eeaa528 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:11.423000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc35736-2405-4bd9-8fcb-56bff0c79aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:11.424012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc35736-2405-4bd9-8fcb-56bff0c79aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:11.424682 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc35736-2405-4bd9-8fcb-56bff0c79aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:11.426298 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc35736-2405-4bd9-8fcb-56bff0c79aa1 None admin] PATH_INFO: `/v3/projects/00cafd6359074bd5ae371a50effe7297` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:11.428711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc35736-2405-4bd9-8fcb-56bff0c79aa1 None admin] RBAC: Authorizing `identity:get_project(project_id=00cafd6359074bd5ae371a50effe7297)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:11.431997 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc35736-2405-4bd9-8fcb-56bff0c79aa1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:11.433981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 216/430] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:11 2022] GET /identity/v3/projects/00cafd6359074bd5ae371a50effe7297 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:11.442498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc35736-2405-4bd9-8fcb-56bff0c79aa1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:11.507541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d446c2-bad2-408b-b738-3b035eeaa528 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:11.543526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef372e48-83c0-47d6-b484-c6f2c2ef25df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:11.543869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef372e48-83c0-47d6-b484-c6f2c2ef25df None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:11.544101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef372e48-83c0-47d6-b484-c6f2c2ef25df None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:11.544363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef372e48-83c0-47d6-b484-c6f2c2ef25df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:11.545096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef372e48-83c0-47d6-b484-c6f2c2ef25df None admin] RBAC: Adding query filter params (group.id=feb35d9f372141cf9eb91fa375ed6239, role.id=dfb71501dfed4ce48cf8cb3559cee9d6, scope.project.id=00cafd6359074bd5ae371a50effe7297) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:11.545398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef372e48-83c0-47d6-b484-c6f2c2ef25df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:11.552590 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef372e48-83c0-47d6-b484-c6f2c2ef25df None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:11.553335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 215/431] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Sep 27 17:11:11 2022] GET /identity/v3/role_assignments?group.id=feb35d9f372141cf9eb91fa375ed6239&scope.project.id=00cafd6359074bd5ae371a50effe7297&role.id=dfb71501dfed4ce48cf8cb3559cee9d6 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:11.643538 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc35736-2405-4bd9-8fcb-56bff0c79aa1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:11.659269 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef372e48-83c0-47d6-b484-c6f2c2ef25df None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:11.688903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:11.705127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d752a-bdf7-49cf-ae34-647ceb0791ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:11.705770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d752a-bdf7-49cf-ae34-647ceb0791ad None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:11.706624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d752a-bdf7-49cf-ae34-647ceb0791ad None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:11.707751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d752a-bdf7-49cf-ae34-647ceb0791ad None admin] PATH_INFO: `/v3/groups/feb35d9f372141cf9eb91fa375ed6239` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:11.711274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d752a-bdf7-49cf-ae34-647ceb0791ad None admin] RBAC: Authorizing `identity:get_group(group_id=feb35d9f372141cf9eb91fa375ed6239)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:11.712758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d752a-bdf7-49cf-ae34-647ceb0791ad None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:11.714424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 217/432] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:11 2022] GET /identity/v3/groups/feb35d9f372141cf9eb91fa375ed6239 => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:11.726348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d752a-bdf7-49cf-ae34-647ceb0791ad None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:11.732391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:11.848899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef372e48-83c0-47d6-b484-c6f2c2ef25df None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:11.895308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d752a-bdf7-49cf-ae34-647ceb0791ad None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:11.895766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c1120-d8aa-411c-acba-6370a83f4d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:11.896614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c1120-d8aa-411c-acba-6370a83f4d5e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:11.897094 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c1120-d8aa-411c-acba-6370a83f4d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:11.897541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934c1120-d8aa-411c-acba-6370a83f4d5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:11.908943 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934c1120-d8aa-411c-acba-6370a83f4d5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-934c1120-d8aa-411c-acba-6370a83f4d5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:11.915297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:11.916465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:11.917498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:11.917498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:11.917498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:11.917498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:11.917498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:11.917498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:11.917498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:11.917498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:11.917498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:11.917498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:11.917498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 27 17:11:11.917498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:11.919376 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 216/433] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:11.925145 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c1120-d8aa-411c-acba-6370a83f4d5e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:11.939301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe26fbc-e93f-4667-89ab-77d7be486340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:11.940053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe26fbc-e93f-4667-89ab-77d7be486340 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:11.941209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe26fbc-e93f-4667-89ab-77d7be486340 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:11.941209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe26fbc-e93f-4667-89ab-77d7be486340 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:11.942149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe26fbc-e93f-4667-89ab-77d7be486340 None admin] RBAC: Adding query filter params (group.id=feb35d9f372141cf9eb91fa375ed6239, role.id=c135afb53eaf43d9a99716160f0ffd25, scope.project.id=00cafd6359074bd5ae371a50effe7297) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:11.942576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe26fbc-e93f-4667-89ab-77d7be486340 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:11.944220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe26fbc-e93f-4667-89ab-77d7be486340 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:11.952215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 218/434] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Sep 27 17:11:11 2022] GET /identity/v3/role_assignments?group.id=feb35d9f372141cf9eb91fa375ed6239&scope.project.id=00cafd6359074bd5ae371a50effe7297&role.id=c135afb53eaf43d9a99716160f0ffd25 => generated 611 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:11.953680 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae214b2-8d00-406d-a9dd-9c76f541b5e7 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:11.954058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae214b2-8d00-406d-a9dd-9c76f541b5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:11.954807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae214b2-8d00-406d-a9dd-9c76f541b5e7 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:11.956809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 219/435] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:11.971787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe129594-ccfe-412b-8150-51f24c18e208 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:11.972254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe129594-ccfe-412b-8150-51f24c18e208 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:11.972627 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe129594-ccfe-412b-8150-51f24c18e208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:11.998373 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-fe129594-ccfe-412b-8150-51f24c18e208 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:12.000078 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fe129594-ccfe-412b-8150-51f24c18e208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:12.026110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:12.053804 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934c1120-d8aa-411c-acba-6370a83f4d5e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:12.089805 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:12.095916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5f6c3-c875-4236-9bad-f605d466bfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:12.097229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5f6c3-c875-4236-9bad-f605d466bfd8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:12.097712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5f6c3-c875-4236-9bad-f605d466bfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:12.098051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5f6c3-c875-4236-9bad-f605d466bfd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:12.098254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 220/436] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:12.103100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5f6c3-c875-4236-9bad-f605d466bfd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:12.103599 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5f6c3-c875-4236-9bad-f605d466bfd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:12.105182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5f6c3-c875-4236-9bad-f605d466bfd8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:12.115937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 217/437] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:11:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:12.123235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe129594-ccfe-412b-8150-51f24c18e208 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:12.240249 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe129594-ccfe-412b-8150-51f24c18e208 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:12.276171 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3769020-b209-452f-89e5-c09a8c703a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:12.276788 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3769020-b209-452f-89e5-c09a8c703a22 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:12.277068 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3769020-b209-452f-89e5-c09a8c703a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:12.277341 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3769020-b209-452f-89e5-c09a8c703a22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:12.358902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:12.361129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3769020-b209-452f-89e5-c09a8c703a22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:12.363298 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3769020-b209-452f-89e5-c09a8c703a22 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:12.382704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5f6c3-c875-4236-9bad-f605d466bfd8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:12.481287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 221/438] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:12.483750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f5447-2d9f-4344-8471-00baf8356b95 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:12.483750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f5447-2d9f-4344-8471-00baf8356b95 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:12.483750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f5447-2d9f-4344-8471-00baf8356b95 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:12.485012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 222/439] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:12.488908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f5447-2d9f-4344-8471-00baf8356b95 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:12.509890 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5f6c3-c875-4236-9bad-f605d466bfd8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:12.558395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14dfd2c-eb73-43b7-bc16-8fde9d40498b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:12.559012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dfd2c-eb73-43b7-bc16-8fde9d40498b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:12.559286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dfd2c-eb73-43b7-bc16-8fde9d40498b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:12.559554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14dfd2c-eb73-43b7-bc16-8fde9d40498b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:12.569766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:12.571033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14dfd2c-eb73-43b7-bc16-8fde9d40498b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d14dfd2c-eb73-43b7-bc16-8fde9d40498b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:12.576502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:12.577429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:12.578352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:12.578352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:12.578352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:12.578352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:12.578352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:12.578352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:12.578352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:12.578352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:12.578352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:12.578352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:12.578352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:12.578352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:12.580718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 218/440] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:11:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:12.581961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781903ba-8060-4d1e-bdc3-aab35403372f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:12.586443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781903ba-8060-4d1e-bdc3-aab35403372f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:12.586913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781903ba-8060-4d1e-bdc3-aab35403372f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:12.602996 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:12.606971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-781903ba-8060-4d1e-bdc3-aab35403372f None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:12.608292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-781903ba-8060-4d1e-bdc3-aab35403372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:12.610415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f5447-2d9f-4344-8471-00baf8356b95 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:12.645476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f960079-c4b3-44fc-94f7-df5ddc4b4b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:12.646118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f960079-c4b3-44fc-94f7-df5ddc4b4b99 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:12.646391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f960079-c4b3-44fc-94f7-df5ddc4b4b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:12.646652 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f960079-c4b3-44fc-94f7-df5ddc4b4b99 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:12.647743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f960079-c4b3-44fc-94f7-df5ddc4b4b99 None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:12.649169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f960079-c4b3-44fc-94f7-df5ddc4b4b99 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:12.650797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 223/441] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:12 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:12.651540 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f960079-c4b3-44fc-94f7-df5ddc4b4b99 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:12.680052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 219/442] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:12.680890 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781903ba-8060-4d1e-bdc3-aab35403372f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:12.758579 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f960079-c4b3-44fc-94f7-df5ddc4b4b99 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:12.788056 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781903ba-8060-4d1e-bdc3-aab35403372f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:12.793769 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb2918-1220-4e6b-aa41-2a301e5c4494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:12.794389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb2918-1220-4e6b-aa41-2a301e5c4494 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:12.794663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb2918-1220-4e6b-aa41-2a301e5c4494 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:12.794926 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb2918-1220-4e6b-aa41-2a301e5c4494 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:12.795903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb2918-1220-4e6b-aa41-2a301e5c4494 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:12.795903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb2918-1220-4e6b-aa41-2a301e5c4494 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:12.797263 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb2918-1220-4e6b-aa41-2a301e5c4494 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:12.805410 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 224/443] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:12.810780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb2918-1220-4e6b-aa41-2a301e5c4494 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:12.815869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:12.825342 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74cda4-8130-4d0b-9165-e19a45fd5f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:12.825965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74cda4-8130-4d0b-9165-e19a45fd5f6f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:12.826264 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74cda4-8130-4d0b-9165-e19a45fd5f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:12.826845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74cda4-8130-4d0b-9165-e19a45fd5f6f None admin] PATH_INFO: `/v3/groups/feb35d9f372141cf9eb91fa375ed6239` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:12.827624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74cda4-8130-4d0b-9165-e19a45fd5f6f None admin] RBAC: Authorizing `identity:get_group(group_id=feb35d9f372141cf9eb91fa375ed6239)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:12.829031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74cda4-8130-4d0b-9165-e19a45fd5f6f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:12.835263 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 220/444] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:12 2022] GET /identity/v3/groups/feb35d9f372141cf9eb91fa375ed6239 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:12.836935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74cda4-8130-4d0b-9165-e19a45fd5f6f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:12.839171 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:12.911892 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb2918-1220-4e6b-aa41-2a301e5c4494 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:12.946139 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74cda4-8130-4d0b-9165-e19a45fd5f6f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:12.946878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb2236-e96c-4e5d-b258-1bdf9d66f564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:12.947529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb2236-e96c-4e5d-b258-1bdf9d66f564 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:12.947807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb2236-e96c-4e5d-b258-1bdf9d66f564 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:12.948071 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb2236-e96c-4e5d-b258-1bdf9d66f564 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:12.980511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef0463-3ab2-4162-bfe7-65c64608c514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:12.981098 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef0463-3ab2-4162-bfe7-65c64608c514 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:12.981627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef0463-3ab2-4162-bfe7-65c64608c514 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:12.981911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef0463-3ab2-4162-bfe7-65c64608c514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:12.982911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef0463-3ab2-4162-bfe7-65c64608c514 None admin] RBAC: Adding query filter params (group.id=feb35d9f372141cf9eb91fa375ed6239, role.id=c135afb53eaf43d9a99716160f0ffd25, scope.project.id=a9f733493ca84fbf8da49edbef53b518) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:12.983213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef0463-3ab2-4162-bfe7-65c64608c514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:12.984629 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acef0463-3ab2-4162-bfe7-65c64608c514 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:12.990710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 221/445] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Sep 27 17:11:12 2022] GET /identity/v3/role_assignments?group.id=feb35d9f372141cf9eb91fa375ed6239&scope.project.id=a9f733493ca84fbf8da49edbef53b518&role.id=c135afb53eaf43d9a99716160f0ffd25 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:12.991321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef0463-3ab2-4162-bfe7-65c64608c514 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:13.021313 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb2236-e96c-4e5d-b258-1bdf9d66f564 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:13.022950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbb2236-e96c-4e5d-b258-1bdf9d66f564 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:13.055603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:13.059142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:13.107598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef0463-3ab2-4162-bfe7-65c64608c514 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:13.149314 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 225/446] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 27 17:11:13.149509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb448ed-037f-4e5e-b842-bc9f305d9f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:13.149509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb448ed-037f-4e5e-b842-bc9f305d9f76 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:13.149509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb448ed-037f-4e5e-b842-bc9f305d9f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:13.149750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb448ed-037f-4e5e-b842-bc9f305d9f76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:13.150503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb2236-e96c-4e5d-b258-1bdf9d66f564 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:13.162117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb448ed-037f-4e5e-b842-bc9f305d9f76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-edb448ed-037f-4e5e-b842-bc9f305d9f76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:13.168244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:13.169173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:13.169964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:13.169964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:13.169964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:13.169964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:13.169964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:13.169964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:13.169964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:13.169964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:13.169964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:13.169964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:13.169964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:13.169964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:13.169964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 222/447] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:13.175152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb448ed-037f-4e5e-b842-bc9f305d9f76 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:13.270978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbb2236-e96c-4e5d-b258-1bdf9d66f564 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:13.296920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb448ed-037f-4e5e-b842-bc9f305d9f76 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:13.303814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:13.319362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805923c-17c8-4ca6-9f86-2fbc0cbfcff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:13.320002 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805923c-17c8-4ca6-9f86-2fbc0cbfcff7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:13.320280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805923c-17c8-4ca6-9f86-2fbc0cbfcff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:13.320802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805923c-17c8-4ca6-9f86-2fbc0cbfcff7 None admin] PATH_INFO: `/v3/projects/a9f733493ca84fbf8da49edbef53b518` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:13.321695 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805923c-17c8-4ca6-9f86-2fbc0cbfcff7 None admin] RBAC: Authorizing `identity:get_project(project_id=a9f733493ca84fbf8da49edbef53b518)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:13.325817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805923c-17c8-4ca6-9f86-2fbc0cbfcff7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:13.325949 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:13.326027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 226/448] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:13 2022] GET /identity/v3/projects/a9f733493ca84fbf8da49edbef53b518 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:13.332910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805923c-17c8-4ca6-9f86-2fbc0cbfcff7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:13.340276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32304c-5081-42e2-8305-f6a7f59f5f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:13.340882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32304c-5081-42e2-8305-f6a7f59f5f58 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:13.341164 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32304c-5081-42e2-8305-f6a7f59f5f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:13.341435 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32304c-5081-42e2-8305-f6a7f59f5f58 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:13.342122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32304c-5081-42e2-8305-f6a7f59f5f58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:13.342430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32304c-5081-42e2-8305-f6a7f59f5f58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:13.343827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32304c-5081-42e2-8305-f6a7f59f5f58 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:13.353400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 223/449] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:13.359361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32304c-5081-42e2-8305-f6a7f59f5f58 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:13.441731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805923c-17c8-4ca6-9f86-2fbc0cbfcff7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:13.459150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32304c-5081-42e2-8305-f6a7f59f5f58 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:13.474701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b676a2-a3b4-4792-ad9b-33c19f0f2fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:13.475355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b676a2-a3b4-4792-ad9b-33c19f0f2fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:13.475851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b676a2-a3b4-4792-ad9b-33c19f0f2fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:13.476327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b676a2-a3b4-4792-ad9b-33c19f0f2fc8 None admin] PATH_INFO: `/v3/groups/feb35d9f372141cf9eb91fa375ed6239` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:13.477474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b676a2-a3b4-4792-ad9b-33c19f0f2fc8 None admin] RBAC: Authorizing `identity:get_group(group_id=feb35d9f372141cf9eb91fa375ed6239)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:13.478958 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b676a2-a3b4-4792-ad9b-33c19f0f2fc8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:13.480526 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 227/450] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:13 2022] GET /identity/v3/groups/feb35d9f372141cf9eb91fa375ed6239 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:13.486519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b676a2-a3b4-4792-ad9b-33c19f0f2fc8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:13.494668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f22d4-d3bd-49ac-8163-588190817f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:13.495325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f22d4-d3bd-49ac-8163-588190817f3c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:13.495607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f22d4-d3bd-49ac-8163-588190817f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:13.495878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f22d4-d3bd-49ac-8163-588190817f3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:13.542880 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:13.547063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:13.561684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f22d4-d3bd-49ac-8163-588190817f3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:13.563158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f22d4-d3bd-49ac-8163-588190817f3c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:13.585620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b676a2-a3b4-4792-ad9b-33c19f0f2fc8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:13.619666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc935d3-b558-4f60-826c-0313f3818fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:13.620270 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc935d3-b558-4f60-826c-0313f3818fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:13.620544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc935d3-b558-4f60-826c-0313f3818fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:13.620815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc935d3-b558-4f60-826c-0313f3818fb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:13.621602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc935d3-b558-4f60-826c-0313f3818fb1 None admin] RBAC: Adding query filter params (group.id=feb35d9f372141cf9eb91fa375ed6239, role.id=dfb71501dfed4ce48cf8cb3559cee9d6, scope.project.id=a9f733493ca84fbf8da49edbef53b518) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:13.621836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc935d3-b558-4f60-826c-0313f3818fb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:13.623286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc935d3-b558-4f60-826c-0313f3818fb1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:13.628923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 228/451] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Sep 27 17:11:13 2022] GET /identity/v3/role_assignments?group.id=feb35d9f372141cf9eb91fa375ed6239&scope.project.id=a9f733493ca84fbf8da49edbef53b518&role.id=dfb71501dfed4ce48cf8cb3559cee9d6 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:13.664206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 224/452] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 27 17:11:13.669962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc935d3-b558-4f60-826c-0313f3818fb1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:13.774534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc935d3-b558-4f60-826c-0313f3818fb1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:13.801825 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:13.813919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34abddd-1ac7-4c81-9723-1f7f71d26f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:13.814582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34abddd-1ac7-4c81-9723-1f7f71d26f2b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:13.814863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34abddd-1ac7-4c81-9723-1f7f71d26f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:13.815138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34abddd-1ac7-4c81-9723-1f7f71d26f2b None admin] PATH_INFO: `/v3/projects/5c93f41c1d2243bfbbfb547f2005e999` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:13.816293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34abddd-1ac7-4c81-9723-1f7f71d26f2b None admin] RBAC: Authorizing `identity:get_project(project_id=5c93f41c1d2243bfbbfb547f2005e999)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:13.817731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34abddd-1ac7-4c81-9723-1f7f71d26f2b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:13.819744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 229/453] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:11:13 2022] GET /identity/v3/projects/5c93f41c1d2243bfbbfb547f2005e999 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:13.824939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f22d4-d3bd-49ac-8163-588190817f3c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:13.908425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:13.928095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f22d4-d3bd-49ac-8163-588190817f3c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:13.962458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698dae0-d046-4315-84e0-cf385d977896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:13.963084 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698dae0-d046-4315-84e0-cf385d977896 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:13.963364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698dae0-d046-4315-84e0-cf385d977896 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:13.963634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698dae0-d046-4315-84e0-cf385d977896 None admin] PATH_INFO: `/v3/groups/7498960280744af982985dff55081bc3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:13.964751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698dae0-d046-4315-84e0-cf385d977896 None admin] RBAC: Authorizing `identity:get_group(group_id=7498960280744af982985dff55081bc3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:13.966236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698dae0-d046-4315-84e0-cf385d977896 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:13.967819 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 225/454] 10.222.0.55 () {56 vars in 1180 bytes} [Tue Sep 27 17:11:13 2022] GET /identity/v3/groups/7498960280744af982985dff55081bc3 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:13.974042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34abddd-1ac7-4c81-9723-1f7f71d26f2b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:14.073579 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34abddd-1ac7-4c81-9723-1f7f71d26f2b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:14.109469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02530696-57dc-48d7-ba90-7da03c1a04c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:14.110092 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02530696-57dc-48d7-ba90-7da03c1a04c4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:14.110371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02530696-57dc-48d7-ba90-7da03c1a04c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:14.110644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02530696-57dc-48d7-ba90-7da03c1a04c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:14.111370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02530696-57dc-48d7-ba90-7da03c1a04c4 None admin] RBAC: Adding query filter params (group.id=7498960280744af982985dff55081bc3, role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=5c93f41c1d2243bfbbfb547f2005e999) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:14.111679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02530696-57dc-48d7-ba90-7da03c1a04c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:14.113727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02530696-57dc-48d7-ba90-7da03c1a04c4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:14.119355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 230/455] 10.222.0.55 () {56 vars in 1376 bytes} [Tue Sep 27 17:11:13 2022] GET /identity/v3/role_assignments?group.id=7498960280744af982985dff55081bc3&scope.project.id=5c93f41c1d2243bfbbfb547f2005e999&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:16.902062 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba8df0-0899-470c-af1f-50a42ceb7f55 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:16.903027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba8df0-0899-470c-af1f-50a42ceb7f55 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:16.903507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba8df0-0899-470c-af1f-50a42ceb7f55 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:16.905122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 226/456] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:16.916198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ead59a-ceec-4194-85bb-109bec7fb48d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:16.916705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ead59a-ceec-4194-85bb-109bec7fb48d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:16.916705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ead59a-ceec-4194-85bb-109bec7fb48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:16.938857 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b3ead59a-ceec-4194-85bb-109bec7fb48d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:16.940155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b3ead59a-ceec-4194-85bb-109bec7fb48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:16.973349 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d329c7-3dc9-43e4-a2c7-d75d5644c9a5 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:16.973624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d329c7-3dc9-43e4-a2c7-d75d5644c9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:16.974086 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d329c7-3dc9-43e4-a2c7-d75d5644c9a5 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:16.975815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 227/457] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:16.983554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55647511-3073-43f6-82c2-42c1e8341e5d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:16.984007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55647511-3073-43f6-82c2-42c1e8341e5d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:16.984419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55647511-3073-43f6-82c2-42c1e8341e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:17.011082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-55647511-3073-43f6-82c2-42c1e8341e5d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:17.012583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-55647511-3073-43f6-82c2-42c1e8341e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:17.034444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:17.045064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 231/458] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:17.080683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892d78f-23f6-4907-8da2-0ad02de59286 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:17.081191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892d78f-23f6-4907-8da2-0ad02de59286 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:17.081609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892d78f-23f6-4907-8da2-0ad02de59286 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:17.083408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 232/459] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:17.087272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 228/460] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:17.088508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4797550-832a-42b3-86cb-a22b85a7fa4c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:17.088785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4797550-832a-42b3-86cb-a22b85a7fa4c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:17.089056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4797550-832a-42b3-86cb-a22b85a7fa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:17.108158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b4797550-832a-42b3-86cb-a22b85a7fa4c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:17.109332 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b4797550-832a-42b3-86cb-a22b85a7fa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:17.177399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 233/461] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:11:17.416714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55647511-3073-43f6-82c2-42c1e8341e5d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:17.454218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4797550-832a-42b3-86cb-a22b85a7fa4c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:17.506394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:17.534097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55647511-3073-43f6-82c2-42c1e8341e5d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:17.565049 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe87c4-295e-49d7-8e11-a015526fca73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:17.565679 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe87c4-295e-49d7-8e11-a015526fca73 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:17.566139 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe87c4-295e-49d7-8e11-a015526fca73 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:17.566428 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe87c4-295e-49d7-8e11-a015526fca73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:17.578103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe87c4-295e-49d7-8e11-a015526fca73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:17.579629 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe87c4-295e-49d7-8e11-a015526fca73 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-3bfe87c4-295e-49d7-8e11-a015526fca73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:17.597305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:17.598319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:17.599186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:17.599186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:17.599186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:17.599186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 229/462] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:17.599439 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe87c4-295e-49d7-8e11-a015526fca73 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:17.611689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4797550-832a-42b3-86cb-a22b85a7fa4c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:17.634846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:17.651259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff643858-17c8-4c2f-ae1d-9bf34e71f98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:17.652069 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff643858-17c8-4c2f-ae1d-9bf34e71f98a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:17.652481 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff643858-17c8-4c2f-ae1d-9bf34e71f98a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:17.652896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff643858-17c8-4c2f-ae1d-9bf34e71f98a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:17.663898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff643858-17c8-4c2f-ae1d-9bf34e71f98a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:17.665430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff643858-17c8-4c2f-ae1d-9bf34e71f98a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ff643858-17c8-4c2f-ae1d-9bf34e71f98a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:17.680394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:17.681283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:17.682894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:17.682894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:17.682894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:17.686362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 234/463] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:17.688453 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922e483-67a5-42c2-97f3-1908b4ac5bca None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:17.688733 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922e483-67a5-42c2-97f3-1908b4ac5bca None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:17.689009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922e483-67a5-42c2-97f3-1908b4ac5bca None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:17.690566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 235/464] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:17.691108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922e483-67a5-42c2-97f3-1908b4ac5bca None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:17.713368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe87c4-295e-49d7-8e11-a015526fca73 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:17.743815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:17.749508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ae734-5e70-4cc3-a172-002887bc319b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:17.750166 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ae734-5e70-4cc3-a172-002887bc319b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:17.750450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ae734-5e70-4cc3-a172-002887bc319b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:17.751214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ae734-5e70-4cc3-a172-002887bc319b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:17.762418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ae734-5e70-4cc3-a172-002887bc319b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:17.763873 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ae734-5e70-4cc3-a172-002887bc319b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-591ae734-5e70-4cc3-a172-002887bc319b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:17.779206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:17.780087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:17.780948 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:17.780948 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:17.780948 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:17.780948 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 230/465] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:17.781158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ae734-5e70-4cc3-a172-002887bc319b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:17.807255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922e483-67a5-42c2-97f3-1908b4ac5bca None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:17.841499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad28536-dc42-4826-9a0b-5884366373e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:17.842390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad28536-dc42-4826-9a0b-5884366373e5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:17.842390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad28536-dc42-4826-9a0b-5884366373e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:17.842666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad28536-dc42-4826-9a0b-5884366373e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:17.843372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad28536-dc42-4826-9a0b-5884366373e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:17.843674 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad28536-dc42-4826-9a0b-5884366373e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:17.845035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad28536-dc42-4826-9a0b-5884366373e5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:17.857284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 236/466] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:17.857284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934b6c2-319c-45b2-90c0-62114b6d895f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:17.857284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934b6c2-319c-45b2-90c0-62114b6d895f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:17.857284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934b6c2-319c-45b2-90c0-62114b6d895f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:17.878157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8934b6c2-319c-45b2-90c0-62114b6d895f None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:17.879350 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8934b6c2-319c-45b2-90c0-62114b6d895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:17.902534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ae734-5e70-4cc3-a172-002887bc319b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:17.909202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:17.938060 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d7097-3b59-4963-b467-77fa3652f650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:17.938742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d7097-3b59-4963-b467-77fa3652f650 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:17.939026 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d7097-3b59-4963-b467-77fa3652f650 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:17.939301 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4d7097-3b59-4963-b467-77fa3652f650 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:17.940478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d7097-3b59-4963-b467-77fa3652f650 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:17.940786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d7097-3b59-4963-b467-77fa3652f650 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:17.942181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4d7097-3b59-4963-b467-77fa3652f650 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:17.958118 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 231/467] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:17.958118 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d7097-3b59-4963-b467-77fa3652f650 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:17.962370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 237/468] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:17.963293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934b6c2-319c-45b2-90c0-62114b6d895f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:18.007517 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:18.063133 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4d7097-3b59-4963-b467-77fa3652f650 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:18.084637 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934b6c2-319c-45b2-90c0-62114b6d895f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:18.098738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fc1b9-b95f-4fca-a7e3-e27c1cdb1549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:18.099408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fc1b9-b95f-4fca-a7e3-e27c1cdb1549 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:18.099690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fc1b9-b95f-4fca-a7e3-e27c1cdb1549 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:18.099962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4fc1b9-b95f-4fca-a7e3-e27c1cdb1549 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:18.100606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fc1b9-b95f-4fca-a7e3-e27c1cdb1549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:18.100907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fc1b9-b95f-4fca-a7e3-e27c1cdb1549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:18.102438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4fc1b9-b95f-4fca-a7e3-e27c1cdb1549 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:18.114302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 232/469] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:18.115176 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fc1b9-b95f-4fca-a7e3-e27c1cdb1549 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:18.123892 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619788bb-daa7-4a4a-b73e-fa939e7ac272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:18.124795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619788bb-daa7-4a4a-b73e-fa939e7ac272 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:18.125384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619788bb-daa7-4a4a-b73e-fa939e7ac272 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:18.125850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619788bb-daa7-4a4a-b73e-fa939e7ac272 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:18.126724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619788bb-daa7-4a4a-b73e-fa939e7ac272 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:18.128893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619788bb-daa7-4a4a-b73e-fa939e7ac272 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:18.157013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 238/470] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:11:17 2022] POST /identity/v3/users => generated 248 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:11:18.162121 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619788bb-daa7-4a4a-b73e-fa939e7ac272 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:18.197648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:18.312267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4fc1b9-b95f-4fca-a7e3-e27c1cdb1549 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:18.322384 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:18.354785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619788bb-daa7-4a4a-b73e-fa939e7ac272 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:18.362775 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c753f3-bd43-4c13-8d22-b0a616c99999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:18.363716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c753f3-bd43-4c13-8d22-b0a616c99999 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:18.364120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c753f3-bd43-4c13-8d22-b0a616c99999 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:18.364583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c753f3-bd43-4c13-8d22-b0a616c99999 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:18.365645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c753f3-bd43-4c13-8d22-b0a616c99999 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:18.368327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c753f3-bd43-4c13-8d22-b0a616c99999 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:18.395732 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 233/471] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:11:18 2022] POST /identity/v3/users => generated 246 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:11:18.395732 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c753f3-bd43-4c13-8d22-b0a616c99999 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:18.397282 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebdf50-f823-4922-9a66-57d3c1566fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:18.397910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebdf50-f823-4922-9a66-57d3c1566fdb None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:18.398233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebdf50-f823-4922-9a66-57d3c1566fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:18.398507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebdf50-f823-4922-9a66-57d3c1566fdb None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:18.399158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebdf50-f823-4922-9a66-57d3c1566fdb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:18.400645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebdf50-f823-4922-9a66-57d3c1566fdb None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:18.426279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 239/472] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:11:18 2022] POST /identity/v3/users => generated 248 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:11:18.427706 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6f0be-918a-4bba-a80d-667230813e9f None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:18.427995 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6f0be-918a-4bba-a80d-667230813e9f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:18.428268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6f0be-918a-4bba-a80d-667230813e9f None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:18.434258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 240/473] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:18.435747 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a003810-cd10-4c3b-85e8-f6f08cbe1a41 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:18.436029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a003810-cd10-4c3b-85e8-f6f08cbe1a41 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:18.436334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a003810-cd10-4c3b-85e8-f6f08cbe1a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:18.457626 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7a003810-cd10-4c3b-85e8-f6f08cbe1a41 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:18.458968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7a003810-cd10-4c3b-85e8-f6f08cbe1a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:18.541599 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c753f3-bd43-4c13-8d22-b0a616c99999 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:18.567433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 241/474] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:11:18.585215 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b7458-bef1-47b2-b730-b2c0b3c03bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:18.585878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b7458-bef1-47b2-b730-b2c0b3c03bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:18.586192 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b7458-bef1-47b2-b730-b2c0b3c03bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:18.586600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b7458-bef1-47b2-b730-b2c0b3c03bb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:18.600216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b7458-bef1-47b2-b730-b2c0b3c03bb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:18.601712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b7458-bef1-47b2-b730-b2c0b3c03bb0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d95b7458-bef1-47b2-b730-b2c0b3c03bb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:18.618783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:18.619770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:18.620614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:18.620614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:18.620614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:18.625168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 234/475] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:18.626667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a003810-cd10-4c3b-85e8-f6f08cbe1a41 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:18.638460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:18.738282 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a003810-cd10-4c3b-85e8-f6f08cbe1a41 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:18.774406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f77fd-091f-40c2-9986-2fcc067af1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:18.775021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f77fd-091f-40c2-9986-2fcc067af1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:18.775299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f77fd-091f-40c2-9986-2fcc067af1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:18.775570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f77fd-091f-40c2-9986-2fcc067af1a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:18.776224 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f77fd-091f-40c2-9986-2fcc067af1a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:18.776525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f77fd-091f-40c2-9986-2fcc067af1a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:18.777895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f77fd-091f-40c2-9986-2fcc067af1a9 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:18.794191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 242/476] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:18.796168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b7458-bef1-47b2-b730-b2c0b3c03bb0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:18.813634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:18.866110 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f77fd-091f-40c2-9986-2fcc067af1a9 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:18.913214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b7458-bef1-47b2-b730-b2c0b3c03bb0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:18.947933 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:18.953820 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb60ca-1dc5-49c2-aff5-82a88cdb3a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:18.953820 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb60ca-1dc5-49c2-aff5-82a88cdb3a67 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:18.953820 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb60ca-1dc5-49c2-aff5-82a88cdb3a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:18.953820 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb60ca-1dc5-49c2-aff5-82a88cdb3a67 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:18.954354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb60ca-1dc5-49c2-aff5-82a88cdb3a67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:18.962115 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb60ca-1dc5-49c2-aff5-82a88cdb3a67 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:18.981716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 235/477] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:11:18 2022] POST /identity/v3/users => generated 249 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:11:19.017215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f77fd-091f-40c2-9986-2fcc067af1a9 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:19.069354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b8053-a475-4a85-8311-e3c88c95590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:19.069974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b8053-a475-4a85-8311-e3c88c95590d None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:19.070279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b8053-a475-4a85-8311-e3c88c95590d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:19.070580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b8053-a475-4a85-8311-e3c88c95590d None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:19.071346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b8053-a475-4a85-8311-e3c88c95590d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:19.072726 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b8053-a475-4a85-8311-e3c88c95590d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:19.086605 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.sql.core [None req-fa5b8053-a475-4a85-8311-e3c88c95590d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 27 17:11:19.086605 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 27 17:11:19.086605 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [parameters: {'id': '354c37f0e4c1402ca4d52fe1bcf2b8e3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 27 17:11:19.086605 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81426) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-fa5b8053-a475-4a85-8311-e3c88c95590d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application conn.query(q) Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application result.read() Sep 27 17:11:19.103511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return ref.to_dict() Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 17:11:19.104446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application next(self.gen) Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application yield resource Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application next(self.gen) Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.session.rollback() Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.force_reraise() Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise self.value Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application session.commit() Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 27 17:11:19.105369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._prepare_impl() Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.session.flush() Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._flush(objects) Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application compat.raise_( Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application flush_context.execute() Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rec.execute(self) Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 27 17:11:19.106536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 17:11:19.107451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application conn.query(q) Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application result.read() Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 17:11:19.108283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [parameters: {'id': '354c37f0e4c1402ca4d52fe1bcf2b8e3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:19.109140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 27 17:11:19.109946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:19.116453 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 243/478] 10.222.0.55 () {60 vars in 1131 bytes} [Tue Sep 27 17:11:18 2022] POST /identity/v3/roles => generated 144 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:19.116616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb60ca-1dc5-49c2-aff5-82a88cdb3a67 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:19.239719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:19.278182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb60ca-1dc5-49c2-aff5-82a88cdb3a67 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:19.323974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7414e-3692-4c63-b777-96eeb007dde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:19.324910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7414e-3692-4c63-b777-96eeb007dde0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:19.325405 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7414e-3692-4c63-b777-96eeb007dde0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:19.325872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7414e-3692-4c63-b777-96eeb007dde0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:19.337119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7414e-3692-4c63-b777-96eeb007dde0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-14f7414e-3692-4c63-b777-96eeb007dde0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:19.342889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:19.343767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:19.344572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:19.344572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:19.344572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:19.344572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:19.344572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:19.344572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:19.344572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:19.344572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:19.344572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:19.344572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:19.344572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 27 17:11:19.344572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:19.347863 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 236/479] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:11:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:19.359026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b8053-a475-4a85-8311-e3c88c95590d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:19.498962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b8053-a475-4a85-8311-e3c88c95590d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:19.538054 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:19.549537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d5697-ee0e-4194-a431-f0bbc7f329e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:19.550173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d5697-ee0e-4194-a431-f0bbc7f329e4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:19.550457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d5697-ee0e-4194-a431-f0bbc7f329e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:19.550732 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d5697-ee0e-4194-a431-f0bbc7f329e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:19.551406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d5697-ee0e-4194-a431-f0bbc7f329e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:19.551709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d5697-ee0e-4194-a431-f0bbc7f329e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:19.553083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d5697-ee0e-4194-a431-f0bbc7f329e4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:19.561606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 244/480] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:20.786227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0345eed-a91c-48a6-9819-b489989b8d20 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:20.786772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0345eed-a91c-48a6-9819-b489989b8d20 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:20.786772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0345eed-a91c-48a6-9819-b489989b8d20 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:20.788334 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 237/481] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:20.793814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6402b06-12c9-4432-9839-af3ec51b78bd None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:20.794269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6402b06-12c9-4432-9839-af3ec51b78bd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:20.794439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6402b06-12c9-4432-9839-af3ec51b78bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:20.816120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e6402b06-12c9-4432-9839-af3ec51b78bd None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:20.817312 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e6402b06-12c9-4432-9839-af3ec51b78bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:20.909044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 245/482] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:21.255165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0345eed-a91c-48a6-9819-b489989b8d20 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:21.369105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c194fc3-9261-4216-b45f-f8687b6ee83b None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:21.369478 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c194fc3-9261-4216-b45f-f8687b6ee83b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:21.369708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c194fc3-9261-4216-b45f-f8687b6ee83b None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:21.371283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 246/483] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:21.377081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd7c77-7298-4d8c-ae1b-8715cf1caf72 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:21.377422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd7c77-7298-4d8c-ae1b-8715cf1caf72 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:21.377673 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd7c77-7298-4d8c-ae1b-8715cf1caf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:21.398656 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3bfd7c77-7298-4d8c-ae1b-8715cf1caf72 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:21.399854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3bfd7c77-7298-4d8c-ae1b-8715cf1caf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:21.417175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0345eed-a91c-48a6-9819-b489989b8d20 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:21.448193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:21.458276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a68b65-58fe-4476-8d13-b1bab4334e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:21.458904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a68b65-58fe-4476-8d13-b1bab4334e54 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:21.459182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a68b65-58fe-4476-8d13-b1bab4334e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:21.459454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a68b65-58fe-4476-8d13-b1bab4334e54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:21.472069 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a68b65-58fe-4476-8d13-b1bab4334e54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-f7a68b65-58fe-4476-8d13-b1bab4334e54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:21.477676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:21.478601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 238/484] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:21.483057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dc924-91f4-4810-9ccc-69e7a09f7758 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:21.483954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dc924-91f4-4810-9ccc-69e7a09f7758 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:21.483954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6dc924-91f4-4810-9ccc-69e7a09f7758 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:21.483954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 239/485] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:21.484795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 247/486] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:11:21.490943 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dc924-91f4-4810-9ccc-69e7a09f7758 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:21.492611 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd9cdd-a9f0-45d8-9cd3-741edfdaaf92 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:21.493084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd9cdd-a9f0-45d8-9cd3-741edfdaaf92 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:21.493515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd9cdd-a9f0-45d8-9cd3-741edfdaaf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:21.514123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-07dd9cdd-a9f0-45d8-9cd3-741edfdaaf92 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:21.515649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-07dd9cdd-a9f0-45d8-9cd3-741edfdaaf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:21.624541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:21.630844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6dc924-91f4-4810-9ccc-69e7a09f7758 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:21.670714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5735031-09e9-451d-9d67-4e70faae1750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:21.671320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5735031-09e9-451d-9d67-4e70faae1750 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:21.671596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5735031-09e9-451d-9d67-4e70faae1750 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:21.671867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5735031-09e9-451d-9d67-4e70faae1750 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:21.672541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5735031-09e9-451d-9d67-4e70faae1750 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:21.672844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5735031-09e9-451d-9d67-4e70faae1750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:21.674821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5735031-09e9-451d-9d67-4e70faae1750 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:21.690957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 248/487] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:11:21.691827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 240/488] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:21.706433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd9cdd-a9f0-45d8-9cd3-741edfdaaf92 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:21.758174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49b06f-3abb-4145-a450-c6f729436938 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:21.758484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49b06f-3abb-4145-a450-c6f729436938 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:21.758719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49b06f-3abb-4145-a450-c6f729436938 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:21.760038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 241/489] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:11:21.765152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa63c2e-7b12-4056-8a61-44412ba777f1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:21.765656 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa63c2e-7b12-4056-8a61-44412ba777f1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:21.766187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa63c2e-7b12-4056-8a61-44412ba777f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:21.785904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0fa63c2e-7b12-4056-8a61-44412ba777f1 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:21.787584 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0fa63c2e-7b12-4056-8a61-44412ba777f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:21.861866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:21.866384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd9cdd-a9f0-45d8-9cd3-741edfdaaf92 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:21.906412 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 242/490] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:11:21.907327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa63c2e-7b12-4056-8a61-44412ba777f1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:21.914984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a5859-ff83-49d6-8256-342f362db96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:21.915853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a5859-ff83-49d6-8256-342f362db96d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:21.916373 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a5859-ff83-49d6-8256-342f362db96d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:21.916831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a5859-ff83-49d6-8256-342f362db96d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:21.929092 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a5859-ff83-49d6-8256-342f362db96d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:21.930906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a5859-ff83-49d6-8256-342f362db96d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ef9a5859-ff83-49d6-8256-342f362db96d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:21.949595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:21.950623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:21.951463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:21.951463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:21.951463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:21.956549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 249/491] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:21.958000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a5859-ff83-49d6-8256-342f362db96d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:22.029356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:22.065265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa63c2e-7b12-4056-8a61-44412ba777f1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:22.103587 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e7bb0-2ec0-4e98-8d31-6ed4e1ceb66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:22.104209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e7bb0-2ec0-4e98-8d31-6ed4e1ceb66c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:22.104481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e7bb0-2ec0-4e98-8d31-6ed4e1ceb66c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:22.104747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e7bb0-2ec0-4e98-8d31-6ed4e1ceb66c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:22.116217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e7bb0-2ec0-4e98-8d31-6ed4e1ceb66c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:22.119729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a5859-ff83-49d6-8256-342f362db96d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0d5e7bb0-2ec0-4e98-8d31-6ed4e1ceb66c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:22.121414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:22.122308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:22.123537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:22.123537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:22.123537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:22.123537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:22.123537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:22.123537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:22.123537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:22.123537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:22.123537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:22.123537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:22.123537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:22.123537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:22.123537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 243/492] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:22.124382 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e7bb0-2ec0-4e98-8d31-6ed4e1ceb66c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:22.171229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:22.172561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb654db-6e2d-4d81-aff0-b8130da69c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:22.172841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb654db-6e2d-4d81-aff0-b8130da69c10 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:22.173502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb654db-6e2d-4d81-aff0-b8130da69c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:22.173773 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb654db-6e2d-4d81-aff0-b8130da69c10 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:22.175405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb654db-6e2d-4d81-aff0-b8130da69c10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:22.175711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb654db-6e2d-4d81-aff0-b8130da69c10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:22.177768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb654db-6e2d-4d81-aff0-b8130da69c10 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:22.195154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 250/493] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:22.196413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb654db-6e2d-4d81-aff0-b8130da69c10 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:22.235344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e7bb0-2ec0-4e98-8d31-6ed4e1ceb66c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:22.271665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8c851-dfad-4192-87d0-098aca68d035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:22.272316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8c851-dfad-4192-87d0-098aca68d035 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:22.272590 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8c851-dfad-4192-87d0-098aca68d035 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:22.272845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8c851-dfad-4192-87d0-098aca68d035 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:22.284181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8c851-dfad-4192-87d0-098aca68d035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-cde8c851-dfad-4192-87d0-098aca68d035 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:22.288875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:22.289785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 244/494] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:22.291835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8c851-dfad-4192-87d0-098aca68d035 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:22.320486 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:22.331495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb654db-6e2d-4d81-aff0-b8130da69c10 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:22.372499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e1375-2f5e-4d8d-b3ac-80177da383f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:22.373172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e1375-2f5e-4d8d-b3ac-80177da383f0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:22.373488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e1375-2f5e-4d8d-b3ac-80177da383f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:22.373712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e1375-2f5e-4d8d-b3ac-80177da383f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:22.375123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e1375-2f5e-4d8d-b3ac-80177da383f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:22.375123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e1375-2f5e-4d8d-b3ac-80177da383f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:22.376252 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e1375-2f5e-4d8d-b3ac-80177da383f0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:22.390255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 251/495] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:22.390255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e1375-2f5e-4d8d-b3ac-80177da383f0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:22.403370 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:22.407489 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8c851-dfad-4192-87d0-098aca68d035 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:22.446686 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5d77e-bc72-4bd9-a543-978084f509cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:22.447340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5d77e-bc72-4bd9-a543-978084f509cc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:22.447641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5d77e-bc72-4bd9-a543-978084f509cc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:22.447911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5d77e-bc72-4bd9-a543-978084f509cc None admin] PATH_INFO: `/v3/users/glance` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:22.456522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5d77e-bc72-4bd9-a543-978084f509cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:22.458045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5d77e-bc72-4bd9-a543-978084f509cc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1cd5d77e-bc72-4bd9-a543-978084f509cc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:22.473196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:22.474202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:22.475097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:22.475097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:22.475097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:22.475097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:22.475097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:22.475097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:22.475097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:22.475097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:22.475097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 27 17:11:22.475097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:22.475097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 245/496] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:22.475097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5d77e-bc72-4bd9-a543-978084f509cc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:22.506881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e1375-2f5e-4d8d-b3ac-80177da383f0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:22.541915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f1e4b-a926-4660-892a-7f9657788e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:22.542483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f1e4b-a926-4660-892a-7f9657788e7f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:22.542757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f1e4b-a926-4660-892a-7f9657788e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:22.543021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f1e4b-a926-4660-892a-7f9657788e7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:22.553712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f1e4b-a926-4660-892a-7f9657788e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-e81f1e4b-a926-4660-892a-7f9657788e7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:22.558427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:22.559426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 252/497] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:22.560519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f1e4b-a926-4660-892a-7f9657788e7f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:22.576098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:22.583546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5d77e-bc72-4bd9-a543-978084f509cc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:22.618480 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23696d0e-9e4d-4032-a025-24003dce402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:22.619076 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23696d0e-9e4d-4032-a025-24003dce402d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:22.619346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23696d0e-9e4d-4032-a025-24003dce402d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:22.619610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23696d0e-9e4d-4032-a025-24003dce402d None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:22.620260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23696d0e-9e4d-4032-a025-24003dce402d None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:22.620554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23696d0e-9e4d-4032-a025-24003dce402d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:22.621916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23696d0e-9e4d-4032-a025-24003dce402d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:22.629707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 246/498] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:22.630503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23696d0e-9e4d-4032-a025-24003dce402d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:22.656517 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:22.663068 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f1e4b-a926-4660-892a-7f9657788e7f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:22.695419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed812ef4-554f-49f1-aef7-31ba64374565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:22.696028 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed812ef4-554f-49f1-aef7-31ba64374565 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:22.696302 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed812ef4-554f-49f1-aef7-31ba64374565 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:22.696568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed812ef4-554f-49f1-aef7-31ba64374565 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:22.706790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed812ef4-554f-49f1-aef7-31ba64374565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:22.708271 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed812ef4-554f-49f1-aef7-31ba64374565 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ed812ef4-554f-49f1-aef7-31ba64374565 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:22.722300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:22.723202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:22.724052 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:22.724052 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:22.724052 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:22.724052 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 253/499] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:22.724598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074ddcc-0843-468d-8d40-5f07805c23d4 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:22.724874 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074ddcc-0843-468d-8d40-5f07805c23d4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:22.725083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074ddcc-0843-468d-8d40-5f07805c23d4 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:22.726308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 254/500] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:22.726987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074ddcc-0843-468d-8d40-5f07805c23d4 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:22.736311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23696d0e-9e4d-4032-a025-24003dce402d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:22.770112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a80876-a52a-44da-9550-dfad34124d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:22.770686 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a80876-a52a-44da-9550-dfad34124d4e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:22.770966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a80876-a52a-44da-9550-dfad34124d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:22.771235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a80876-a52a-44da-9550-dfad34124d4e None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:22.771983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a80876-a52a-44da-9550-dfad34124d4e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:22.772280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a80876-a52a-44da-9550-dfad34124d4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:22.773702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a80876-a52a-44da-9550-dfad34124d4e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:22.791698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:22.809746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 247/501] 10.222.0.55 () {56 vars in 1137 bytes} [Tue Sep 27 17:11:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:22.810609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a80876-a52a-44da-9550-dfad34124d4e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:22.827784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074ddcc-0843-468d-8d40-5f07805c23d4 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:22.862870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0d3bd-24dc-46a6-8924-7e979cfaaa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:22.863469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0d3bd-24dc-46a6-8924-7e979cfaaa3b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:22.863746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0d3bd-24dc-46a6-8924-7e979cfaaa3b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:22.864014 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0d3bd-24dc-46a6-8924-7e979cfaaa3b None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:22.864672 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0d3bd-24dc-46a6-8924-7e979cfaaa3b None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:22.864973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0d3bd-24dc-46a6-8924-7e979cfaaa3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:22.866358 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b0d3bd-24dc-46a6-8924-7e979cfaaa3b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:22.873913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 255/502] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:22.874736 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0d3bd-24dc-46a6-8924-7e979cfaaa3b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:22.891379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:22.911080 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a80876-a52a-44da-9550-dfad34124d4e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:22.944964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cdae5-a7c4-4a7b-8799-f379e0418c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:22.945574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cdae5-a7c4-4a7b-8799-f379e0418c19 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:22.945847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cdae5-a7c4-4a7b-8799-f379e0418c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:22.946206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cdae5-a7c4-4a7b-8799-f379e0418c19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:22.956528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1cdae5-a7c4-4a7b-8799-f379e0418c19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:22.957985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1cdae5-a7c4-4a7b-8799-f379e0418c19 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-aa1cdae5-a7c4-4a7b-8799-f379e0418c19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:22.972749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:22.973847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:22.974761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:22.974761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:22.974761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:22.974925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0d3bd-24dc-46a6-8924-7e979cfaaa3b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:22.975004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 248/503] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:22.975238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec8f0d-ba25-4897-b238-9395b8b7b183 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:22.975513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec8f0d-ba25-4897-b238-9395b8b7b183 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:22.975778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec8f0d-ba25-4897-b238-9395b8b7b183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:22.992940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2dec8f0d-ba25-4897-b238-9395b8b7b183 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:22.994140 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2dec8f0d-ba25-4897-b238-9395b8b7b183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:23.003077 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:23.011039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b354301-3035-4382-b6c5-7715c7be7d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:23.011638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354301-3035-4382-b6c5-7715c7be7d49 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:23.011913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354301-3035-4382-b6c5-7715c7be7d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:23.012181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b354301-3035-4382-b6c5-7715c7be7d49 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:23.012831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b354301-3035-4382-b6c5-7715c7be7d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:23.013133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b354301-3035-4382-b6c5-7715c7be7d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:23.014548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b354301-3035-4382-b6c5-7715c7be7d49 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:23.026390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 256/504] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:23.027191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b354301-3035-4382-b6c5-7715c7be7d49 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:23.064231 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 249/505] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:23.065545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec8f0d-ba25-4897-b238-9395b8b7b183 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:23.131951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:23.138932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b354301-3035-4382-b6c5-7715c7be7d49 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:23.174279 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec8f0d-ba25-4897-b238-9395b8b7b183 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:23.175149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737294d-a8ef-480c-a530-f04e2f9e0082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:23.175783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737294d-a8ef-480c-a530-f04e2f9e0082 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:23.176065 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737294d-a8ef-480c-a530-f04e2f9e0082 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:23.176338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737294d-a8ef-480c-a530-f04e2f9e0082 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:23.211932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202cb1c-f713-4094-b4ea-0f73d714e94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:23.213519 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202cb1c-f713-4094-b4ea-0f73d714e94f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:23.213991 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202cb1c-f713-4094-b4ea-0f73d714e94f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:23.214508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202cb1c-f713-4094-b4ea-0f73d714e94f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:23.226702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202cb1c-f713-4094-b4ea-0f73d714e94f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:23.228139 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202cb1c-f713-4094-b4ea-0f73d714e94f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:23.234281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-a202cb1c-f713-4094-b4ea-0f73d714e94f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:23.243690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:23.244585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:23.245439 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:23.245439 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:23.245439 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:23.249322 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737294d-a8ef-480c-a530-f04e2f9e0082 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:23.249534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 250/506] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:23.250305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202cb1c-f713-4094-b4ea-0f73d714e94f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:23.250694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737294d-a8ef-480c-a530-f04e2f9e0082 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:23.358843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202cb1c-f713-4094-b4ea-0f73d714e94f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:23.364094 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 257/507] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:23.364094 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737294d-a8ef-480c-a530-f04e2f9e0082 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:23.373400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:23.394855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e0311-4709-4d97-8b9a-824c4a8811ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:23.395459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0311-4709-4d97-8b9a-824c4a8811ea None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:23.395741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0311-4709-4d97-8b9a-824c4a8811ea None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:23.396011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0311-4709-4d97-8b9a-824c4a8811ea None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:23.396655 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e0311-4709-4d97-8b9a-824c4a8811ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:23.396952 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e0311-4709-4d97-8b9a-824c4a8811ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:23.398345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e0311-4709-4d97-8b9a-824c4a8811ea None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:23.410887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 251/508] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:23.411670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e0311-4709-4d97-8b9a-824c4a8811ea None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:23.446904 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:23.460253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737294d-a8ef-480c-a530-f04e2f9e0082 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:23.493613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18331cdb-1fa8-40c8-b69b-bcf189732b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:23.494258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18331cdb-1fa8-40c8-b69b-bcf189732b2f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:23.494537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18331cdb-1fa8-40c8-b69b-bcf189732b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:23.494806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18331cdb-1fa8-40c8-b69b-bcf189732b2f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:23.501777 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18331cdb-1fa8-40c8-b69b-bcf189732b2f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:23.503304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18331cdb-1fa8-40c8-b69b-bcf189732b2f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:23.507647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e0311-4709-4d97-8b9a-824c4a8811ea None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-18331cdb-1fa8-40c8-b69b-bcf189732b2f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:23.515877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:23.516772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:23.517643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:23.517643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:23.517643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:23.517643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:23.517643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:23.517643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:23.517643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:23.517643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:23.517643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:11:23.517643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:23.517643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 258/509] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Sep 27 17:11:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:23.519151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18331cdb-1fa8-40c8-b69b-bcf189732b2f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:23.544227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b209fd9-d18a-4359-b056-67889dc0c9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:23.544819 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b209fd9-d18a-4359-b056-67889dc0c9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:23.545092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b209fd9-d18a-4359-b056-67889dc0c9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:23.545366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b209fd9-d18a-4359-b056-67889dc0c9a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:23.546001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b209fd9-d18a-4359-b056-67889dc0c9a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:23.546326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b209fd9-d18a-4359-b056-67889dc0c9a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:23.547679 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b209fd9-d18a-4359-b056-67889dc0c9a9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:23.558558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 252/510] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:23.559338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b209fd9-d18a-4359-b056-67889dc0c9a9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:23.597383 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:23.617728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18331cdb-1fa8-40c8-b69b-bcf189732b2f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:23.650683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26037572-6d63-471e-bebc-289e69cf7984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:23.651306 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26037572-6d63-471e-bebc-289e69cf7984 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:23.651605 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26037572-6d63-471e-bebc-289e69cf7984 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:23.651860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26037572-6d63-471e-bebc-289e69cf7984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:23.662822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26037572-6d63-471e-bebc-289e69cf7984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:23.664064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b209fd9-d18a-4359-b056-67889dc0c9a9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:23.664289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26037572-6d63-471e-bebc-289e69cf7984 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:23.672179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-26037572-6d63-471e-bebc-289e69cf7984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:23.678903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:23.679955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:23.681055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:23.681055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:23.681055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:23.682433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 259/511] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:23.683205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26037572-6d63-471e-bebc-289e69cf7984 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:23.697341 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c95c5f-47cf-408d-94f3-873c90a008af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:23.697969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c95c5f-47cf-408d-94f3-873c90a008af None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:23.698295 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c95c5f-47cf-408d-94f3-873c90a008af None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:23.698566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c95c5f-47cf-408d-94f3-873c90a008af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:23.709581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c95c5f-47cf-408d-94f3-873c90a008af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:23.711058 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c95c5f-47cf-408d-94f3-873c90a008af None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-59c95c5f-47cf-408d-94f3-873c90a008af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:23.724881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:23.725846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:23.726755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:23.726755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:23.726755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:23.726755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 253/512] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:23.726981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c95c5f-47cf-408d-94f3-873c90a008af None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:23.784442 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26037572-6d63-471e-bebc-289e69cf7984 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:23.810498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:23.818460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd695a8f-5510-4fcb-babb-a1e9e35a3fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:23.819080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd695a8f-5510-4fcb-babb-a1e9e35a3fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:23.819355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd695a8f-5510-4fcb-babb-a1e9e35a3fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:23.819622 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd695a8f-5510-4fcb-babb-a1e9e35a3fd5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:23.826111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c95c5f-47cf-408d-94f3-873c90a008af None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:23.827505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd695a8f-5510-4fcb-babb-a1e9e35a3fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:23.828941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd695a8f-5510-4fcb-babb-a1e9e35a3fd5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-fd695a8f-5510-4fcb-babb-a1e9e35a3fd5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:23.839867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:23.840743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:23.841645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:23.841645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:23.841645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:23.841645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:23.841645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:23.841645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:23.841645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:23.841645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:23.841645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 17:11:23.841645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:23.841645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 260/513] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:23.842432 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd695a8f-5510-4fcb-babb-a1e9e35a3fd5 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:23.864046 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d8646-86dd-4e93-bcf1-38baee66c40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:23.864581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d8646-86dd-4e93-bcf1-38baee66c40b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:23.864855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d8646-86dd-4e93-bcf1-38baee66c40b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:23.865123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d8646-86dd-4e93-bcf1-38baee66c40b None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:23.865850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d8646-86dd-4e93-bcf1-38baee66c40b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:23.866183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d8646-86dd-4e93-bcf1-38baee66c40b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:23.867600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d8646-86dd-4e93-bcf1-38baee66c40b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:23.885222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 254/514] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Sep 27 17:11:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:23.886226 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d8646-86dd-4e93-bcf1-38baee66c40b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:23.899872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:23.942206 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd695a8f-5510-4fcb-babb-a1e9e35a3fd5 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:23.974656 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a40f0-af98-4689-b947-a37df61e36b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:23.975276 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a40f0-af98-4689-b947-a37df61e36b0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:23.975555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a40f0-af98-4689-b947-a37df61e36b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:23.975828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a40f0-af98-4689-b947-a37df61e36b0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:23.983280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a40f0-af98-4689-b947-a37df61e36b0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:23.984724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a40f0-af98-4689-b947-a37df61e36b0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:23.987776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d8646-86dd-4e93-bcf1-38baee66c40b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-aa4a40f0-af98-4689-b947-a37df61e36b0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:23.995563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:23.996610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:24.003076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:24.003076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:24.003076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:24.003076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:24.003076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:24.003076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:24.003076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:24.003076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:24.003076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 27 17:11:24.003076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:24.003076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 261/515] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:24.003076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a40f0-af98-4689-b947-a37df61e36b0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:24.022206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5aa54-59b3-4c16-8df7-2b9dd9997d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:24.022801 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5aa54-59b3-4c16-8df7-2b9dd9997d82 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:24.023073 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5aa54-59b3-4c16-8df7-2b9dd9997d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:24.023340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5aa54-59b3-4c16-8df7-2b9dd9997d82 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:24.023975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5aa54-59b3-4c16-8df7-2b9dd9997d82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:24.024259 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5aa54-59b3-4c16-8df7-2b9dd9997d82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:24.025601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5aa54-59b3-4c16-8df7-2b9dd9997d82 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:24.037270 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 255/516] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:24.038047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5aa54-59b3-4c16-8df7-2b9dd9997d82 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:24.043544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:24.098437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a40f0-af98-4689-b947-a37df61e36b0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:24.117617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:24.132556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fd7de-6e54-46ae-af29-92a0c3e3dd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:24.133153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fd7de-6e54-46ae-af29-92a0c3e3dd16 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:24.133430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fd7de-6e54-46ae-af29-92a0c3e3dd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:24.133701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fd7de-6e54-46ae-af29-92a0c3e3dd16 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:24.134372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fd7de-6e54-46ae-af29-92a0c3e3dd16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:24.134674 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fd7de-6e54-46ae-af29-92a0c3e3dd16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:24.136027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fd7de-6e54-46ae-af29-92a0c3e3dd16 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:24.138424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5aa54-59b3-4c16-8df7-2b9dd9997d82 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:24.147734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 262/517] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:24.148822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fd7de-6e54-46ae-af29-92a0c3e3dd16 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:24.175732 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904984d7-1ffa-4385-bcaf-251ac8f4b1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:24.176321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904984d7-1ffa-4385-bcaf-251ac8f4b1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:24.176598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904984d7-1ffa-4385-bcaf-251ac8f4b1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:24.176867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904984d7-1ffa-4385-bcaf-251ac8f4b1d3 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:24.177582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904984d7-1ffa-4385-bcaf-251ac8f4b1d3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:24.177886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904984d7-1ffa-4385-bcaf-251ac8f4b1d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:24.179323 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904984d7-1ffa-4385-bcaf-251ac8f4b1d3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:24.196134 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 256/518] 10.222.0.55 () {56 vars in 1137 bytes} [Tue Sep 27 17:11:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:24.196923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904984d7-1ffa-4385-bcaf-251ac8f4b1d3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:24.245841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fd7de-6e54-46ae-af29-92a0c3e3dd16 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:24.256364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:24.280104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283768d6-3a36-481b-94a8-4be3848488fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:24.280696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283768d6-3a36-481b-94a8-4be3848488fb None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:24.280971 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283768d6-3a36-481b-94a8-4be3848488fb None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:24.281242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283768d6-3a36-481b-94a8-4be3848488fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:24.294765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904984d7-1ffa-4385-bcaf-251ac8f4b1d3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:24.329723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:24.331054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5ffcc-20ce-4421-8803-0e37ed190990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:24.331597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5ffcc-20ce-4421-8803-0e37ed190990 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:24.331845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5ffcc-20ce-4421-8803-0e37ed190990 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:24.332141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5ffcc-20ce-4421-8803-0e37ed190990 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:24.332831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5ffcc-20ce-4421-8803-0e37ed190990 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:24.333138 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5ffcc-20ce-4421-8803-0e37ed190990 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:24.334652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5ffcc-20ce-4421-8803-0e37ed190990 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:24.349011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283768d6-3a36-481b-94a8-4be3848488fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:24.350484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283768d6-3a36-481b-94a8-4be3848488fb None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:24.351269 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 257/519] 10.222.0.55 () {56 vars in 1139 bytes} [Tue Sep 27 17:11:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:24.352037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5ffcc-20ce-4421-8803-0e37ed190990 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:24.450893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5ffcc-20ce-4421-8803-0e37ed190990 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:24.451782 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 263/520] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:24.452621 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283768d6-3a36-481b-94a8-4be3848488fb None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:24.475199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:24.489623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807135f-b63f-48c8-a6c2-ccb824c3b8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:24.490252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807135f-b63f-48c8-a6c2-ccb824c3b8cd None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:24.490530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807135f-b63f-48c8-a6c2-ccb824c3b8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:24.490799 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807135f-b63f-48c8-a6c2-ccb824c3b8cd None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:24.491468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807135f-b63f-48c8-a6c2-ccb824c3b8cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:24.492882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807135f-b63f-48c8-a6c2-ccb824c3b8cd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:24.512894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 258/521] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:11:24 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:11:24.513681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807135f-b63f-48c8-a6c2-ccb824c3b8cd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:24.541447 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:24.563325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283768d6-3a36-481b-94a8-4be3848488fb None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:24.602517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97662ac-6a79-4591-9bff-345d02038d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:24.603085 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97662ac-6a79-4591-9bff-345d02038d04 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:24.603359 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97662ac-6a79-4591-9bff-345d02038d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:24.603626 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97662ac-6a79-4591-9bff-345d02038d04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:24.613575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97662ac-6a79-4591-9bff-345d02038d04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:24.615071 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97662ac-6a79-4591-9bff-345d02038d04 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a97662ac-6a79-4591-9bff-345d02038d04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:24.628303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:24.629434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:24.630390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:24.630390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:24.630390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:24.630390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:24.630390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:24.630390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:24.630390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:24.630390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:24.630390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 264/522] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:24.630859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97662ac-6a79-4591-9bff-345d02038d04 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:24.653661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807135f-b63f-48c8-a6c2-ccb824c3b8cd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:24.695180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecb68a-3857-45d5-ac60-7ed63b58aaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:24.695783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecb68a-3857-45d5-ac60-7ed63b58aaa2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:24.696060 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecb68a-3857-45d5-ac60-7ed63b58aaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:24.696333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecb68a-3857-45d5-ac60-7ed63b58aaa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:24.748363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97662ac-6a79-4591-9bff-345d02038d04 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:24.749123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:24.777698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeecb68a-3857-45d5-ac60-7ed63b58aaa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:24.779519 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeecb68a-3857-45d5-ac60-7ed63b58aaa2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:24.784399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecf869-8ba4-41df-afac-960608c9b21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:24.785000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecf869-8ba4-41df-afac-960608c9b21c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:24.785273 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecf869-8ba4-41df-afac-960608c9b21c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:24.785553 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecf869-8ba4-41df-afac-960608c9b21c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:24.796493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:24.857689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ecf869-8ba4-41df-afac-960608c9b21c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:24.859180 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ecf869-8ba4-41df-afac-960608c9b21c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:24.884154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 259/523] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:24.885005 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecb68a-3857-45d5-ac60-7ed63b58aaa2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:24.972153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 265/524] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:24.972153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecf869-8ba4-41df-afac-960608c9b21c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:24.973810 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:25.000613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecb68a-3857-45d5-ac60-7ed63b58aaa2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:25.011560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:25.038110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b2d88-89ba-4b32-8d27-436849b8d355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:25.038945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b2d88-89ba-4b32-8d27-436849b8d355 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:25.039324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b2d88-89ba-4b32-8d27-436849b8d355 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:25.039697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b2d88-89ba-4b32-8d27-436849b8d355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:25.050621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b2d88-89ba-4b32-8d27-436849b8d355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:25.052184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b2d88-89ba-4b32-8d27-436849b8d355 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-538b2d88-89ba-4b32-8d27-436849b8d355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:25.066905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:25.068038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:25.069153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:25.069153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:25.069153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:25.072574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 260/525] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:25.073372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b2d88-89ba-4b32-8d27-436849b8d355 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:25.080587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecf869-8ba4-41df-afac-960608c9b21c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:25.119728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ba026-04c5-4681-8f2f-2cc885e92d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:25.120996 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ba026-04c5-4681-8f2f-2cc885e92d7d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:25.121425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ba026-04c5-4681-8f2f-2cc885e92d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:25.121846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ba026-04c5-4681-8f2f-2cc885e92d7d None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:25.122698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ba026-04c5-4681-8f2f-2cc885e92d7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:25.123134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ba026-04c5-4681-8f2f-2cc885e92d7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:25.124705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ba026-04c5-4681-8f2f-2cc885e92d7d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:25.142849 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 266/526] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:25.148543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ba026-04c5-4681-8f2f-2cc885e92d7d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:25.175914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b2d88-89ba-4b32-8d27-436849b8d355 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:25.205392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:25.215367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225051c-2c15-4c1e-83a2-9e4d41f5afa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:25.215991 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225051c-2c15-4c1e-83a2-9e4d41f5afa0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:25.216275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225051c-2c15-4c1e-83a2-9e4d41f5afa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:25.216547 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225051c-2c15-4c1e-83a2-9e4d41f5afa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:25.227985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225051c-2c15-4c1e-83a2-9e4d41f5afa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:25.229444 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225051c-2c15-4c1e-83a2-9e4d41f5afa0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-3225051c-2c15-4c1e-83a2-9e4d41f5afa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:25.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:25.245516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:25.246493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:25.246493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:25.246493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:25.246493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 261/527] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:25.246728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225051c-2c15-4c1e-83a2-9e4d41f5afa0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:25.259487 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:25.262837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ba026-04c5-4681-8f2f-2cc885e92d7d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:25.298349 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dc74a-b134-473a-83eb-e9964df6ba70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:25.299003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dc74a-b134-473a-83eb-e9964df6ba70 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:25.299276 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dc74a-b134-473a-83eb-e9964df6ba70 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:25.299541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dc74a-b134-473a-83eb-e9964df6ba70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:25.312698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420dc74a-b134-473a-83eb-e9964df6ba70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:25.314178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420dc74a-b134-473a-83eb-e9964df6ba70 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-420dc74a-b134-473a-83eb-e9964df6ba70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:25.329699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:25.330970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:25.331812 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:25.331812 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:25.331812 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:25.331812 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 267/528] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:25.331812 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dc74a-b134-473a-83eb-e9964df6ba70 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:25.371218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225051c-2c15-4c1e-83a2-9e4d41f5afa0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:25.405395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc12b8-d5c1-48a2-8474-eb8d8206d1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:25.406053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc12b8-d5c1-48a2-8474-eb8d8206d1bb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:25.406355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc12b8-d5c1-48a2-8474-eb8d8206d1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:25.406671 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc12b8-d5c1-48a2-8474-eb8d8206d1bb None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:25.407260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc12b8-d5c1-48a2-8474-eb8d8206d1bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:25.408668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc12b8-d5c1-48a2-8474-eb8d8206d1bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:25.409429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc12b8-d5c1-48a2-8474-eb8d8206d1bb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:25.426289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 262/529] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:25.426289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc12b8-d5c1-48a2-8474-eb8d8206d1bb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:25.447269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dc74a-b134-473a-83eb-e9964df6ba70 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:25.472623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:25.481763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f39c1-fc0a-45a1-8f9d-972dacf1dc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:25.482386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f39c1-fc0a-45a1-8f9d-972dacf1dc31 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:25.482659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f39c1-fc0a-45a1-8f9d-972dacf1dc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:25.482923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f39c1-fc0a-45a1-8f9d-972dacf1dc31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:25.483647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f39c1-fc0a-45a1-8f9d-972dacf1dc31 None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=6cc89c1a7e684612a73ec9fef6bda780) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:25.483949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f39c1-fc0a-45a1-8f9d-972dacf1dc31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:25.485369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f39c1-fc0a-45a1-8f9d-972dacf1dc31 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:25.491545 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 268/530] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:25 2022] GET /identity/v3/role_assignments?user.id=6cc89c1a7e684612a73ec9fef6bda780&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:25.503453 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f39c1-fc0a-45a1-8f9d-972dacf1dc31 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:25.527771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:25.550538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc12b8-d5c1-48a2-8474-eb8d8206d1bb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:25.603382 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd38d7-8510-4423-bc0d-3d410c534044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:25.604933 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd38d7-8510-4423-bc0d-3d410c534044 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:25.605451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd38d7-8510-4423-bc0d-3d410c534044 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:25.605927 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd38d7-8510-4423-bc0d-3d410c534044 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:25.607086 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd38d7-8510-4423-bc0d-3d410c534044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:25.608235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd38d7-8510-4423-bc0d-3d410c534044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:25.610256 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd38d7-8510-4423-bc0d-3d410c534044 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:25.621572 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f39c1-fc0a-45a1-8f9d-972dacf1dc31 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:25.629169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 263/531] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:25.630508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd38d7-8510-4423-bc0d-3d410c534044 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:25.664771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fcee6-6583-4332-9926-c9fff2472457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:25.665837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fcee6-6583-4332-9926-c9fff2472457 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:25.666162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fcee6-6583-4332-9926-c9fff2472457 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:25.666528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358fcee6-6583-4332-9926-c9fff2472457 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:25.667083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fcee6-6583-4332-9926-c9fff2472457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:25.667379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fcee6-6583-4332-9926-c9fff2472457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:25.668740 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358fcee6-6583-4332-9926-c9fff2472457 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:25.687430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 269/532] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:25.687430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fcee6-6583-4332-9926-c9fff2472457 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:25.751413 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:25.756163 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd38d7-8510-4423-bc0d-3d410c534044 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:25.790303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:25.797364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e0adb-d0fe-47be-92f5-185b1694c73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:25.797928 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e0adb-d0fe-47be-92f5-185b1694c73c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:25.798231 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e0adb-d0fe-47be-92f5-185b1694c73c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:25.798500 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e0adb-d0fe-47be-92f5-185b1694c73c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:25.806125 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e0adb-d0fe-47be-92f5-185b1694c73c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:25.807742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4e0adb-d0fe-47be-92f5-185b1694c73c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-fa4e0adb-d0fe-47be-92f5-185b1694c73c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:25.819921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:25.820836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:25.821703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:25.821703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:25.821703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:25.821703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:25.821703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:25.821703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:25.821703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:25.821703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:25.824016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358fcee6-6583-4332-9926-c9fff2472457 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:25.825892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 264/533] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:25.826739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e0adb-d0fe-47be-92f5-185b1694c73c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:25.869656 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac54775-d3f8-4b77-a772-919fd745c241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:25.870714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac54775-d3f8-4b77-a772-919fd745c241 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:25.871542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac54775-d3f8-4b77-a772-919fd745c241 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:25.872125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac54775-d3f8-4b77-a772-919fd745c241 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:25.881154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac54775-d3f8-4b77-a772-919fd745c241 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:25.883490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac54775-d3f8-4b77-a772-919fd745c241 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-5ac54775-d3f8-4b77-a772-919fd745c241 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:25.898934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:25.899801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:25.900655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:25.900655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:25.900655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:25.900655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:25.900655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:25.900655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:25.900655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:25.900655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:25.903894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 270/534] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:25.904946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac54775-d3f8-4b77-a772-919fd745c241 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:25.933738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e0adb-d0fe-47be-92f5-185b1694c73c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:25.971556 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c57e08-6b29-4324-be75-63bc99386d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:25.972338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c57e08-6b29-4324-be75-63bc99386d97 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:25.972620 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c57e08-6b29-4324-be75-63bc99386d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:25.972888 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c57e08-6b29-4324-be75-63bc99386d97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:25.979902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c57e08-6b29-4324-be75-63bc99386d97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:25.981665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c57e08-6b29-4324-be75-63bc99386d97 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-68c57e08-6b29-4324-be75-63bc99386d97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:25.993849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:25.994782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:25.995710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:25.995710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:25.995710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:25.995710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:25.995710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:25.995710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:25.995710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:25.995710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:25.995710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 265/535] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:25.995710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c57e08-6b29-4324-be75-63bc99386d97 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:26.035581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:26.038184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac54775-d3f8-4b77-a772-919fd745c241 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:26.076464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd267ea3-2239-4b45-806f-c82c0f8702ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:26.077179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd267ea3-2239-4b45-806f-c82c0f8702ab None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:26.077303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd267ea3-2239-4b45-806f-c82c0f8702ab None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:26.077501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd267ea3-2239-4b45-806f-c82c0f8702ab None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:26.078212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd267ea3-2239-4b45-806f-c82c0f8702ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:26.078513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd267ea3-2239-4b45-806f-c82c0f8702ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:26.079943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd267ea3-2239-4b45-806f-c82c0f8702ab None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:26.082820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:26.098339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 271/536] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:26.099587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd267ea3-2239-4b45-806f-c82c0f8702ab None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:26.145545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c57e08-6b29-4324-be75-63bc99386d97 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:26.185437 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301b9d0-5192-48e5-b298-6f4ea5021be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:26.186040 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301b9d0-5192-48e5-b298-6f4ea5021be6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:26.186363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301b9d0-5192-48e5-b298-6f4ea5021be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:26.186628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301b9d0-5192-48e5-b298-6f4ea5021be6 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:26.187472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301b9d0-5192-48e5-b298-6f4ea5021be6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:26.187670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301b9d0-5192-48e5-b298-6f4ea5021be6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:26.189175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301b9d0-5192-48e5-b298-6f4ea5021be6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:26.209210 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 266/537] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:26.210171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301b9d0-5192-48e5-b298-6f4ea5021be6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:26.218984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd267ea3-2239-4b45-806f-c82c0f8702ab None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:26.262369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad275d9-10d9-4e59-a5e4-aa87a7c47368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:26.262964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad275d9-10d9-4e59-a5e4-aa87a7c47368 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:26.263234 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad275d9-10d9-4e59-a5e4-aa87a7c47368 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:26.263515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad275d9-10d9-4e59-a5e4-aa87a7c47368 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:26.264208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad275d9-10d9-4e59-a5e4-aa87a7c47368 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:26.264507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad275d9-10d9-4e59-a5e4-aa87a7c47368 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:26.265922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad275d9-10d9-4e59-a5e4-aa87a7c47368 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:26.284049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 272/538] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:26.284049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad275d9-10d9-4e59-a5e4-aa87a7c47368 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:26.319584 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:26.344542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301b9d0-5192-48e5-b298-6f4ea5021be6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:26.353672 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:26.385566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f267eb0-b018-40ee-b4a9-06956217d1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:26.386790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f267eb0-b018-40ee-b4a9-06956217d1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:26.387781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f267eb0-b018-40ee-b4a9-06956217d1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:26.388205 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f267eb0-b018-40ee-b4a9-06956217d1b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:26.389410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f267eb0-b018-40ee-b4a9-06956217d1b2 None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=5537185f42474cfd86fca70fa8beddd2) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:26.389867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f267eb0-b018-40ee-b4a9-06956217d1b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:26.392180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f267eb0-b018-40ee-b4a9-06956217d1b2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:26.403983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 267/539] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:26 2022] GET /identity/v3/role_assignments?user.id=5537185f42474cfd86fca70fa8beddd2&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:26.405072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f267eb0-b018-40ee-b4a9-06956217d1b2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:26.417907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad275d9-10d9-4e59-a5e4-aa87a7c47368 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:26.491806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf1736-c1d5-47d4-bf3b-6086140b6588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:26.492419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf1736-c1d5-47d4-bf3b-6086140b6588 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:26.492688 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf1736-c1d5-47d4-bf3b-6086140b6588 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:26.492950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf1736-c1d5-47d4-bf3b-6086140b6588 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:26.493697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf1736-c1d5-47d4-bf3b-6086140b6588 None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=528981e41f0b436b82989d62c713d993) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:26.494001 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf1736-c1d5-47d4-bf3b-6086140b6588 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:26.496862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf1736-c1d5-47d4-bf3b-6086140b6588 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:26.510826 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 273/540] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:26 2022] GET /identity/v3/role_assignments?user.id=528981e41f0b436b82989d62c713d993&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:26.568296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f267eb0-b018-40ee-b4a9-06956217d1b2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:26.611748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a184cd-4169-4792-8e43-f992f1fd1a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:26.612282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a184cd-4169-4792-8e43-f992f1fd1a08 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:26.612555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a184cd-4169-4792-8e43-f992f1fd1a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:26.612824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a184cd-4169-4792-8e43-f992f1fd1a08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:26.613663 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a184cd-4169-4792-8e43-f992f1fd1a08 None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=5d906452199844149966c9c86db4684e) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:26.613973 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a184cd-4169-4792-8e43-f992f1fd1a08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:26.615581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a184cd-4169-4792-8e43-f992f1fd1a08 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:26.622867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 268/541] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:26 2022] GET /identity/v3/role_assignments?user.id=5d906452199844149966c9c86db4684e&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:26.749507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671bab0-c393-4b04-be88-5e8de679345d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:26.749849 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671bab0-c393-4b04-be88-5e8de679345d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:26.750041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671bab0-c393-4b04-be88-5e8de679345d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:26.751797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 274/542] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:26.758632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7737be-54d2-458a-862c-5025545da9cc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:26.758941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7737be-54d2-458a-862c-5025545da9cc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:26.759188 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7737be-54d2-458a-862c-5025545da9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:26.778980 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ea7737be-54d2-458a-862c-5025545da9cc None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:26.780216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ea7737be-54d2-458a-862c-5025545da9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:26.782519 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:26.868017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 269/543] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:27.202594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671bab0-c393-4b04-be88-5e8de679345d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:27.311505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671bab0-c393-4b04-be88-5e8de679345d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:27.333332 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:27.351985 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91ab0e-afba-466d-88f7-7b11ae21bf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:27.352618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91ab0e-afba-466d-88f7-7b11ae21bf81 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:27.352893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91ab0e-afba-466d-88f7-7b11ae21bf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:27.353162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91ab0e-afba-466d-88f7-7b11ae21bf81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:27.369740 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91ab0e-afba-466d-88f7-7b11ae21bf81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-2f91ab0e-afba-466d-88f7-7b11ae21bf81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:27.374544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:27.375452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:27.376244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:27.376244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:27.376244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:27.376244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:27.376244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:27.376244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:27.376244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:27.376244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:27.376244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:27.376244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:27.376244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:27.376244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:27.376244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 275/544] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:27.381816 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7737be-54d2-458a-862c-5025545da9cc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:27.510906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7737be-54d2-458a-862c-5025545da9cc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:27.550233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be1077-6177-4482-93c9-651e5a2ceff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:27.550842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be1077-6177-4482-93c9-651e5a2ceff6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:27.551119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be1077-6177-4482-93c9-651e5a2ceff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:27.551392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be1077-6177-4482-93c9-651e5a2ceff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:27.552054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be1077-6177-4482-93c9-651e5a2ceff6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:27.552353 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be1077-6177-4482-93c9-651e5a2ceff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:27.553719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be1077-6177-4482-93c9-651e5a2ceff6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:27.559183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:27.570001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 270/545] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:27.575665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91ab0e-afba-466d-88f7-7b11ae21bf81 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:27.711530 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91ab0e-afba-466d-88f7-7b11ae21bf81 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:27.747902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a6e7d-13af-4f9d-9361-0770bee29d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:27.748528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a6e7d-13af-4f9d-9361-0770bee29d38 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:27.748806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a6e7d-13af-4f9d-9361-0770bee29d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:27.749072 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a6e7d-13af-4f9d-9361-0770bee29d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:27.759996 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a6e7d-13af-4f9d-9361-0770bee29d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:27.761516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a6e7d-13af-4f9d-9361-0770bee29d38 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-fe7a6e7d-13af-4f9d-9361-0770bee29d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:27.787815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:27.788785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:27.789620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:27.789620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:27.789620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:27.789620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 276/546] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:27.794621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be1077-6177-4482-93c9-651e5a2ceff6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:27.938806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be1077-6177-4482-93c9-651e5a2ceff6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:27.975615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f8b99-bb18-41b8-8305-c019c88b9658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:27.976228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f8b99-bb18-41b8-8305-c019c88b9658 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:27.976507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f8b99-bb18-41b8-8305-c019c88b9658 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:27.976779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f8b99-bb18-41b8-8305-c019c88b9658 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:27.977425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f8b99-bb18-41b8-8305-c019c88b9658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:27.977724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f8b99-bb18-41b8-8305-c019c88b9658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:27.979124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f8b99-bb18-41b8-8305-c019c88b9658 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:27.996312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 271/547] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:28.001558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a6e7d-13af-4f9d-9361-0770bee29d38 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:28.014284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449c1a9-7f00-4ce3-bd04-3a1c27aef902 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:28.014907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449c1a9-7f00-4ce3-bd04-3a1c27aef902 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:28.015367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449c1a9-7f00-4ce3-bd04-3a1c27aef902 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:28.017129 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 272/548] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:28.022752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2355845-be5a-45e4-8fed-a41e63ad24d0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:28.023385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2355845-be5a-45e4-8fed-a41e63ad24d0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:28.023847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2355845-be5a-45e4-8fed-a41e63ad24d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:28.043455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a2355845-be5a-45e4-8fed-a41e63ad24d0 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:28.045015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a2355845-be5a-45e4-8fed-a41e63ad24d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:28.077053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:28.138427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:28.155067 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 273/549] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:11:28.201695 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a6e7d-13af-4f9d-9361-0770bee29d38 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:28.243362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f06c9b-c5e6-4214-9d15-c31a490afe42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:28.243973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f06c9b-c5e6-4214-9d15-c31a490afe42 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:28.244236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f06c9b-c5e6-4214-9d15-c31a490afe42 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:28.244503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f06c9b-c5e6-4214-9d15-c31a490afe42 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:28.252251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f06c9b-c5e6-4214-9d15-c31a490afe42 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:28.253898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f06c9b-c5e6-4214-9d15-c31a490afe42 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-63f06c9b-c5e6-4214-9d15-c31a490afe42 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:28.265185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:28.266774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:28.268392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:28.268392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:28.268392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:28.268392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:28.268392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:28.268392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:28.268392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:28.268392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:28.268392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 27 17:11:28.268392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:28.268392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 277/550] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:28.275193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2355845-be5a-45e4-8fed-a41e63ad24d0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:28.394975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2355845-be5a-45e4-8fed-a41e63ad24d0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:28.444502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d213fc-3448-42c1-92a9-2b68b8dec19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:28.445113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d213fc-3448-42c1-92a9-2b68b8dec19f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:28.445392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d213fc-3448-42c1-92a9-2b68b8dec19f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:28.445665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d213fc-3448-42c1-92a9-2b68b8dec19f None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:28.446430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d213fc-3448-42c1-92a9-2b68b8dec19f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:28.446732 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d213fc-3448-42c1-92a9-2b68b8dec19f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:28.448224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d213fc-3448-42c1-92a9-2b68b8dec19f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:28.465793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 274/551] 10.222.0.55 () {56 vars in 1135 bytes} [Tue Sep 27 17:11:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:28.471766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f06c9b-c5e6-4214-9d15-c31a490afe42 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:28.612592 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:28.619315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d213fc-3448-42c1-92a9-2b68b8dec19f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:28.628897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f06c9b-c5e6-4214-9d15-c31a490afe42 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:28.672524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:28.702649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7a219-9a4a-4d45-8b4a-fefd01f3ad8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:28.703758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7a219-9a4a-4d45-8b4a-fefd01f3ad8e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:28.706193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7a219-9a4a-4d45-8b4a-fefd01f3ad8e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:28.706971 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7a219-9a4a-4d45-8b4a-fefd01f3ad8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:28.784484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d213fc-3448-42c1-92a9-2b68b8dec19f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:28.835369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7a219-9a4a-4d45-8b4a-fefd01f3ad8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:28.837233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7a219-9a4a-4d45-8b4a-fefd01f3ad8e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:28.840938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957dc97-a0d0-4e73-bee2-ff0c53c2a4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:28.846391 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957dc97-a0d0-4e73-bee2-ff0c53c2a4ec None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:28.846989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957dc97-a0d0-4e73-bee2-ff0c53c2a4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:28.847469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957dc97-a0d0-4e73-bee2-ff0c53c2a4ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:28.868272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957dc97-a0d0-4e73-bee2-ff0c53c2a4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c957dc97-a0d0-4e73-bee2-ff0c53c2a4ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:28.873516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:28.874574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:28.875399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:28.875399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:28.875399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:28.875399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:28.875399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:28.875399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:28.875399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:28.875399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:28.875399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:28.875399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:28.875399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:28.875399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:28.887192 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 275/552] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:28.895002 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957dc97-a0d0-4e73-bee2-ff0c53c2a4ec None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:28.976862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:28.999505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 278/553] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:29.005775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7a219-9a4a-4d45-8b4a-fefd01f3ad8e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:29.057984 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957dc97-a0d0-4e73-bee2-ff0c53c2a4ec None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:29.079111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:29.123343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d72d0-ffe3-4edf-bafa-a576a25f1ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:29.123865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d72d0-ffe3-4edf-bafa-a576a25f1ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:29.124145 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d72d0-ffe3-4edf-bafa-a576a25f1ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:29.124410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d72d0-ffe3-4edf-bafa-a576a25f1ba9 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:29.125349 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d72d0-ffe3-4edf-bafa-a576a25f1ba9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:29.125654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d72d0-ffe3-4edf-bafa-a576a25f1ba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:29.128578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d72d0-ffe3-4edf-bafa-a576a25f1ba9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:29.138404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 276/554] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:29.139876 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1af8e-382c-462e-9a48-5e37733055b2 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:29.140163 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1af8e-382c-462e-9a48-5e37733055b2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:29.140551 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1af8e-382c-462e-9a48-5e37733055b2 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:29.142192 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 277/555] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:29.145824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1af8e-382c-462e-9a48-5e37733055b2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:29.176393 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7a219-9a4a-4d45-8b4a-fefd01f3ad8e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:29.214784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67667c-5c68-465e-b284-db3fc1dfe009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:29.215638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67667c-5c68-465e-b284-db3fc1dfe009 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:29.216013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67667c-5c68-465e-b284-db3fc1dfe009 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:29.216384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67667c-5c68-465e-b284-db3fc1dfe009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:29.232494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67667c-5c68-465e-b284-db3fc1dfe009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:29.234120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67667c-5c68-465e-b284-db3fc1dfe009 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ad67667c-5c68-465e-b284-db3fc1dfe009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:29.256966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:29.258417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:29.259316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:29.259316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:29.259316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:29.259316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 279/556] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:29.259316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04428a4-073f-4ccb-a382-c536afcb15cf None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:29.259601 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04428a4-073f-4ccb-a382-c536afcb15cf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:29.260197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04428a4-073f-4ccb-a382-c536afcb15cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:29.281873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e04428a4-073f-4ccb-a382-c536afcb15cf None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:29.283635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e04428a4-073f-4ccb-a382-c536afcb15cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:29.288413 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1af8e-382c-462e-9a48-5e37733055b2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:29.330277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8047aa-e9de-4dd5-beed-74b5652adc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:29.330906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8047aa-e9de-4dd5-beed-74b5652adc59 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:29.331184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8047aa-e9de-4dd5-beed-74b5652adc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:29.331454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8047aa-e9de-4dd5-beed-74b5652adc59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:29.332672 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:29.343070 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8047aa-e9de-4dd5-beed-74b5652adc59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:29.344526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8047aa-e9de-4dd5-beed-74b5652adc59 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ef8047aa-e9de-4dd5-beed-74b5652adc59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:29.362358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:29.363883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:29.364752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:29.364752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:29.364752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:29.368189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 278/557] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:29.369089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8047aa-e9de-4dd5-beed-74b5652adc59 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:29.372071 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 280/558] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:29.379810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7d3f3-85a6-402b-ab9b-3450733e7bd7 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:29.379810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7d3f3-85a6-402b-ab9b-3450733e7bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:29.379810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7d3f3-85a6-402b-ab9b-3450733e7bd7 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:29.379810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 281/559] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:29.379810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7d3f3-85a6-402b-ab9b-3450733e7bd7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:29.410452 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:29.506860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8047aa-e9de-4dd5-beed-74b5652adc59 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:29.535742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7d3f3-85a6-402b-ab9b-3450733e7bd7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:29.548629 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4bba0d-dd71-4cb7-b668-7bcee21639e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:29.549426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bba0d-dd71-4cb7-b668-7bcee21639e7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:29.549701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bba0d-dd71-4cb7-b668-7bcee21639e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:29.550153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bba0d-dd71-4cb7-b668-7bcee21639e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:29.551167 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4bba0d-dd71-4cb7-b668-7bcee21639e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:29.551478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4bba0d-dd71-4cb7-b668-7bcee21639e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:29.552832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4bba0d-dd71-4cb7-b668-7bcee21639e7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:29.564913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 279/560] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:29.567494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d375d-3b78-42b7-b9ac-c9d9fd63d8f2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:29.568003 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d375d-3b78-42b7-b9ac-c9d9fd63d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:29.568488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d375d-3b78-42b7-b9ac-c9d9fd63d8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:29.578129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb52bcc-d42c-446d-9661-5c162f751973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:29.578490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb52bcc-d42c-446d-9661-5c162f751973 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:29.578705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb52bcc-d42c-446d-9661-5c162f751973 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:29.578975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb52bcc-d42c-446d-9661-5c162f751973 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:29.579622 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb52bcc-d42c-446d-9661-5c162f751973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:29.580456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb52bcc-d42c-446d-9661-5c162f751973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:29.581827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb52bcc-d42c-446d-9661-5c162f751973 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:29.588900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-302d375d-3b78-42b7-b9ac-c9d9fd63d8f2 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:29.590273 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-302d375d-3b78-42b7-b9ac-c9d9fd63d8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:29.598872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 282/561] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:29.599890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da407321-e555-474f-babb-ae413fb60cc9 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:29.600176 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da407321-e555-474f-babb-ae413fb60cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:29.600448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da407321-e555-474f-babb-ae413fb60cc9 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:29.601893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 283/562] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:29.606172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da407321-e555-474f-babb-ae413fb60cc9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:29.646668 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:29.672510 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 280/563] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:29.673360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d375d-3b78-42b7-b9ac-c9d9fd63d8f2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:29.686917 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:29.730316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da407321-e555-474f-babb-ae413fb60cc9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:29.774380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904c53e-78a1-4bbf-b984-b1787d936ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:29.775300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904c53e-78a1-4bbf-b984-b1787d936ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:29.775745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904c53e-78a1-4bbf-b984-b1787d936ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:29.775890 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d375d-3b78-42b7-b9ac-c9d9fd63d8f2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:29.776297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904c53e-78a1-4bbf-b984-b1787d936ea6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:29.784170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7904c53e-78a1-4bbf-b984-b1787d936ea6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:29.785946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7904c53e-78a1-4bbf-b984-b1787d936ea6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-7904c53e-78a1-4bbf-b984-b1787d936ea6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:29.800602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:29.801721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:29.803048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:29.803048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:29.803048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:29.803048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:29.803048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:29.803048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:29.803048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:29.803048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:29.804921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 284/564] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:29.806382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c862b-76d3-413c-871b-d3daddcee41f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:29.806665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c862b-76d3-413c-871b-d3daddcee41f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:29.806938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2c862b-76d3-413c-871b-d3daddcee41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:29.815274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf90db-f11a-4049-baba-d615d4f23e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:29.816137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf90db-f11a-4049-baba-d615d4f23e18 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:29.816415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf90db-f11a-4049-baba-d615d4f23e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:29.816685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf90db-f11a-4049-baba-d615d4f23e18 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:29.823942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf90db-f11a-4049-baba-d615d4f23e18 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:29.825370 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf90db-f11a-4049-baba-d615d4f23e18 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:29.826740 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ab2c862b-76d3-413c-871b-d3daddcee41f None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:29.828262 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ab2c862b-76d3-413c-871b-d3daddcee41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-73cf90db-f11a-4049-baba-d615d4f23e18 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:29.837539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:29.838688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:29.839666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:29.839666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:29.839666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:29.839666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:29.839666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:29.839666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:29.839666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:29.839666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:29.839666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 27 17:11:29.839666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:29.839666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 281/565] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:29.843255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf90db-f11a-4049-baba-d615d4f23e18 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:29.908604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 285/566] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:29.909659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c862b-76d3-413c-871b-d3daddcee41f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:29.935398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:29.947426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:29.975253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf90db-f11a-4049-baba-d615d4f23e18 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:30.012997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df1949-bda8-4f61-80c1-0bb1f25da544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:30.013616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df1949-bda8-4f61-80c1-0bb1f25da544 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:30.013889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df1949-bda8-4f61-80c1-0bb1f25da544 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:30.014180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df1949-bda8-4f61-80c1-0bb1f25da544 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:30.026811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df1949-bda8-4f61-80c1-0bb1f25da544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:30.031543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2c862b-76d3-413c-871b-d3daddcee41f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d3df1949-bda8-4f61-80c1-0bb1f25da544 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:30.032082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:30.033233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 282/567] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:30.034199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df1949-bda8-4f61-80c1-0bb1f25da544 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:30.076660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debb14a-0950-4b67-83f7-618905b26baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:30.077502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debb14a-0950-4b67-83f7-618905b26baf None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:30.077925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debb14a-0950-4b67-83f7-618905b26baf None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:30.078370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debb14a-0950-4b67-83f7-618905b26baf None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:30.079232 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debb14a-0950-4b67-83f7-618905b26baf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:30.079691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debb14a-0950-4b67-83f7-618905b26baf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:30.081991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debb14a-0950-4b67-83f7-618905b26baf None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:30.100876 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 286/568] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:30.101935 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debb14a-0950-4b67-83f7-618905b26baf None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:30.140360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df1949-bda8-4f61-80c1-0bb1f25da544 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:30.172582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:30.179337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c0bcb-69e9-4d66-8bb8-41d0e8f48d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:30.180192 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c0bcb-69e9-4d66-8bb8-41d0e8f48d56 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:30.180690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c0bcb-69e9-4d66-8bb8-41d0e8f48d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:30.181163 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c0bcb-69e9-4d66-8bb8-41d0e8f48d56 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:30.182138 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c0bcb-69e9-4d66-8bb8-41d0e8f48d56 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:30.182658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c0bcb-69e9-4d66-8bb8-41d0e8f48d56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:30.184297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c0bcb-69e9-4d66-8bb8-41d0e8f48d56 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:30.195326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:30.203060 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 283/569] 10.222.0.55 () {56 vars in 1137 bytes} [Tue Sep 27 17:11:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:30.203874 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c0bcb-69e9-4d66-8bb8-41d0e8f48d56 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:30.212673 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debb14a-0950-4b67-83f7-618905b26baf None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:30.246893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a6551-838e-4b63-8a4b-45c3fb7d63d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:30.247508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a6551-838e-4b63-8a4b-45c3fb7d63d7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:30.247781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a6551-838e-4b63-8a4b-45c3fb7d63d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:30.248044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a6551-838e-4b63-8a4b-45c3fb7d63d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:30.258308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a6551-838e-4b63-8a4b-45c3fb7d63d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-939a6551-838e-4b63-8a4b-45c3fb7d63d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:30.263342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:30.264366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 287/570] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:30.265203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a6551-838e-4b63-8a4b-45c3fb7d63d7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:30.307007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c0bcb-69e9-4d66-8bb8-41d0e8f48d56 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:30.340645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebcbca-bc7c-4780-b768-5f6bade1d325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:30.341238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebcbca-bc7c-4780-b768-5f6bade1d325 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:30.341510 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebcbca-bc7c-4780-b768-5f6bade1d325 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:30.341776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ebcbca-bc7c-4780-b768-5f6bade1d325 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:30.342464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebcbca-bc7c-4780-b768-5f6bade1d325 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:30.342759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebcbca-bc7c-4780-b768-5f6bade1d325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:30.344121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ebcbca-bc7c-4780-b768-5f6bade1d325 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:30.351681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 284/571] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:30.352447 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebcbca-bc7c-4780-b768-5f6bade1d325 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:30.363421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a6551-838e-4b63-8a4b-45c3fb7d63d7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:30.396702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f641f-2f26-4674-919a-bdc56171afd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:30.397294 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f641f-2f26-4674-919a-bdc56171afd8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:30.397572 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f641f-2f26-4674-919a-bdc56171afd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:30.397842 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f641f-2f26-4674-919a-bdc56171afd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:30.398602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f641f-2f26-4674-919a-bdc56171afd8 None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=39073006a0054b56bb1bba46d2d0768e) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:30.398911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f641f-2f26-4674-919a-bdc56171afd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:30.400325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f641f-2f26-4674-919a-bdc56171afd8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:30.404878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:30.414295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 288/572] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:30 2022] GET /identity/v3/role_assignments?user.id=39073006a0054b56bb1bba46d2d0768e&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:30.414295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f641f-2f26-4674-919a-bdc56171afd8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:30.426443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:30.460371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ebcbca-bc7c-4780-b768-5f6bade1d325 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:30.498837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea615b26-cf8a-4239-8cb2-05e44669e379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:30.499483 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea615b26-cf8a-4239-8cb2-05e44669e379 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:30.499764 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea615b26-cf8a-4239-8cb2-05e44669e379 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:30.500038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea615b26-cf8a-4239-8cb2-05e44669e379 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:30.513050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea615b26-cf8a-4239-8cb2-05e44669e379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ea615b26-cf8a-4239-8cb2-05e44669e379 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:30.518532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:30.519605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 285/573] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:30.520454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea615b26-cf8a-4239-8cb2-05e44669e379 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:30.535567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f641f-2f26-4674-919a-bdc56171afd8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:30.574107 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379013f-62e2-4dc3-a67f-6a1ad845fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:30.574743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379013f-62e2-4dc3-a67f-6a1ad845fa82 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:30.575025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379013f-62e2-4dc3-a67f-6a1ad845fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:30.575295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379013f-62e2-4dc3-a67f-6a1ad845fa82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:30.587483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379013f-62e2-4dc3-a67f-6a1ad845fa82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:30.588565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3379013f-62e2-4dc3-a67f-6a1ad845fa82 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-3379013f-62e2-4dc3-a67f-6a1ad845fa82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:30.603389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:30.604371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:30.605261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:30.605261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:30.605261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:30.605261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 289/574] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:30.605261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379013f-62e2-4dc3-a67f-6a1ad845fa82 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:30.639372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea615b26-cf8a-4239-8cb2-05e44669e379 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:30.653426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:30.669075 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:30.681506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4214df5-86c3-4e57-8ba4-6f0c03e4fabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:30.682059 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4214df5-86c3-4e57-8ba4-6f0c03e4fabf None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:30.682364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4214df5-86c3-4e57-8ba4-6f0c03e4fabf None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:30.682635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4214df5-86c3-4e57-8ba4-6f0c03e4fabf None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:30.683297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4214df5-86c3-4e57-8ba4-6f0c03e4fabf None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:30.683597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4214df5-86c3-4e57-8ba4-6f0c03e4fabf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:30.684964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4214df5-86c3-4e57-8ba4-6f0c03e4fabf None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:30.699121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 286/575] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:30.699121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4214df5-86c3-4e57-8ba4-6f0c03e4fabf None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:30.716701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379013f-62e2-4dc3-a67f-6a1ad845fa82 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:30.751085 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29426fd2-29d0-4418-9e6c-3441562e8a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:30.752249 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29426fd2-29d0-4418-9e6c-3441562e8a51 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:30.752528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29426fd2-29d0-4418-9e6c-3441562e8a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:30.752798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29426fd2-29d0-4418-9e6c-3441562e8a51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:30.764345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29426fd2-29d0-4418-9e6c-3441562e8a51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:30.765802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29426fd2-29d0-4418-9e6c-3441562e8a51 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-29426fd2-29d0-4418-9e6c-3441562e8a51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:30.783113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:30.784366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:30.785472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:30.785472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:30.785472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:30.788930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 290/576] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:30.789676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29426fd2-29d0-4418-9e6c-3441562e8a51 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:30.807827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4214df5-86c3-4e57-8ba4-6f0c03e4fabf None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:30.843403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917992e-bfa9-4dbb-b7ac-46905934d9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:30.844169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917992e-bfa9-4dbb-b7ac-46905934d9df None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:30.844443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917992e-bfa9-4dbb-b7ac-46905934d9df None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:30.844716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917992e-bfa9-4dbb-b7ac-46905934d9df None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:30.845398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917992e-bfa9-4dbb-b7ac-46905934d9df None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:30.845693 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917992e-bfa9-4dbb-b7ac-46905934d9df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:30.847206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917992e-bfa9-4dbb-b7ac-46905934d9df None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:30.858467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 287/577] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:30.859340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917992e-bfa9-4dbb-b7ac-46905934d9df None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:30.897233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29426fd2-29d0-4418-9e6c-3441562e8a51 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:30.901769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:30.909970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:30.930839 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fbf4b-54f2-4467-9b59-30d5cf5008f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:30.931435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fbf4b-54f2-4467-9b59-30d5cf5008f0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:30.931757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fbf4b-54f2-4467-9b59-30d5cf5008f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:30.931973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fbf4b-54f2-4467-9b59-30d5cf5008f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:30.932601 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fbf4b-54f2-4467-9b59-30d5cf5008f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:30.932901 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fbf4b-54f2-4467-9b59-30d5cf5008f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:30.934272 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fbf4b-54f2-4467-9b59-30d5cf5008f0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:30.950767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 291/578] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:30.950767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fbf4b-54f2-4467-9b59-30d5cf5008f0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:30.977355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917992e-bfa9-4dbb-b7ac-46905934d9df None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:31.011555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f596e83-23ec-41d8-9e62-4a39604d8f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:31.012338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f596e83-23ec-41d8-9e62-4a39604d8f36 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:31.012939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f596e83-23ec-41d8-9e62-4a39604d8f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:31.013215 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f596e83-23ec-41d8-9e62-4a39604d8f36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:31.026425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f596e83-23ec-41d8-9e62-4a39604d8f36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:31.027867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f596e83-23ec-41d8-9e62-4a39604d8f36 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-9f596e83-23ec-41d8-9e62-4a39604d8f36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:31.044261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:31.045222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:31.051175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:31.051175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:31.051175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:31.051175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 288/579] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:31.051175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f596e83-23ec-41d8-9e62-4a39604d8f36 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:31.058453 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fbf4b-54f2-4467-9b59-30d5cf5008f0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:31.093231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab4f53-1199-46fa-b418-84be71c08e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:31.093986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab4f53-1199-46fa-b418-84be71c08e28 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:31.094392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab4f53-1199-46fa-b418-84be71c08e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:31.094673 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab4f53-1199-46fa-b418-84be71c08e28 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:31.095349 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab4f53-1199-46fa-b418-84be71c08e28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:31.096188 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab4f53-1199-46fa-b418-84be71c08e28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:31.097565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab4f53-1199-46fa-b418-84be71c08e28 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:31.110130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 292/580] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:31.114969 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab4f53-1199-46fa-b418-84be71c08e28 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:31.145633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:31.148344 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:31.161617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f596e83-23ec-41d8-9e62-4a39604d8f36 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:31.198930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2dd67-dd76-4277-8020-df7f87a6ef7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:31.199482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2dd67-dd76-4277-8020-df7f87a6ef7f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:31.199761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2dd67-dd76-4277-8020-df7f87a6ef7f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:31.200037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2dd67-dd76-4277-8020-df7f87a6ef7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:31.211992 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2dd67-dd76-4277-8020-df7f87a6ef7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:31.213429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2dd67-dd76-4277-8020-df7f87a6ef7f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:31.220946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab4f53-1199-46fa-b418-84be71c08e28 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-f4f2dd67-dd76-4277-8020-df7f87a6ef7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:31.228855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:31.229766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:31.230792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:31.230792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:31.230792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:31.230792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 289/581] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:31.230792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2dd67-dd76-4277-8020-df7f87a6ef7f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:31.263501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d34edb-486a-435a-afa9-f2f85ddbab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:31.264250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d34edb-486a-435a-afa9-f2f85ddbab94 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:31.264591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d34edb-486a-435a-afa9-f2f85ddbab94 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:31.264896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d34edb-486a-435a-afa9-f2f85ddbab94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:31.272441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d34edb-486a-435a-afa9-f2f85ddbab94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:31.273920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d34edb-486a-435a-afa9-f2f85ddbab94 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-29d34edb-486a-435a-afa9-f2f85ddbab94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:31.286653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:31.287593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:31.288469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:31.288469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:31.288469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:31.288469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:31.288469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:31.288469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:31.288469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:31.288469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:31.288469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 293/582] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:31.288964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d34edb-486a-435a-afa9-f2f85ddbab94 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:31.343289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2dd67-dd76-4277-8020-df7f87a6ef7f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:31.377932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c7bed-1d8e-4f1f-bd91-935969339da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:31.378554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c7bed-1d8e-4f1f-bd91-935969339da9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:31.378823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c7bed-1d8e-4f1f-bd91-935969339da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:31.379088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c7bed-1d8e-4f1f-bd91-935969339da9 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:31.379721 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c7bed-1d8e-4f1f-bd91-935969339da9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:31.380023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c7bed-1d8e-4f1f-bd91-935969339da9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:31.381344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c7bed-1d8e-4f1f-bd91-935969339da9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:31.387641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:31.390769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:31.395683 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 290/583] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:31.396470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c7bed-1d8e-4f1f-bd91-935969339da9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:31.403012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d34edb-486a-435a-afa9-f2f85ddbab94 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:31.442092 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67e23d-229a-425e-a79a-40d787cc44c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:31.442742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67e23d-229a-425e-a79a-40d787cc44c4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:31.443340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67e23d-229a-425e-a79a-40d787cc44c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:31.443619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67e23d-229a-425e-a79a-40d787cc44c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:31.453691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67e23d-229a-425e-a79a-40d787cc44c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:31.455660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67e23d-229a-425e-a79a-40d787cc44c4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-bf67e23d-229a-425e-a79a-40d787cc44c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:31.470216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:31.471644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:31.472589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:31.472589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:31.472589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:31.472589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:31.472589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:31.472589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:31.472589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:31.472589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:31.472589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:11:31.472589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:31.476973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 294/584] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Sep 27 17:11:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:31.477742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67e23d-229a-425e-a79a-40d787cc44c4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:31.503858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c7bed-1d8e-4f1f-bd91-935969339da9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:31.540729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43a6f8-d2dc-4b54-a7d4-a80cb624ed11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:31.541332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43a6f8-d2dc-4b54-a7d4-a80cb624ed11 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:31.541604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43a6f8-d2dc-4b54-a7d4-a80cb624ed11 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:31.541869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43a6f8-d2dc-4b54-a7d4-a80cb624ed11 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:31.542536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43a6f8-d2dc-4b54-a7d4-a80cb624ed11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:31.542831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43a6f8-d2dc-4b54-a7d4-a80cb624ed11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:31.544169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43a6f8-d2dc-4b54-a7d4-a80cb624ed11 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:31.555868 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 291/585] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:31.556651 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43a6f8-d2dc-4b54-a7d4-a80cb624ed11 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:31.595950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67e23d-229a-425e-a79a-40d787cc44c4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:31.614697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:31.635894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938de2a0-a153-4277-9321-4cd44c82bce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:31.636505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938de2a0-a153-4277-9321-4cd44c82bce6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:31.636770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938de2a0-a153-4277-9321-4cd44c82bce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:31.637046 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938de2a0-a153-4277-9321-4cd44c82bce6 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:31.637742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938de2a0-a153-4277-9321-4cd44c82bce6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:31.638046 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938de2a0-a153-4277-9321-4cd44c82bce6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:31.639506 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938de2a0-a153-4277-9321-4cd44c82bce6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:31.656809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 295/586] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:31.656809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938de2a0-a153-4277-9321-4cd44c82bce6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:31.666596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43a6f8-d2dc-4b54-a7d4-a80cb624ed11 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:31.669671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:31.702009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7de632-aba6-40e0-a3e3-4d7fa48d21d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:31.702679 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de632-aba6-40e0-a3e3-4d7fa48d21d4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:31.702950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de632-aba6-40e0-a3e3-4d7fa48d21d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:31.703371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de632-aba6-40e0-a3e3-4d7fa48d21d4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:31.711106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7de632-aba6-40e0-a3e3-4d7fa48d21d4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:31.712484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7de632-aba6-40e0-a3e3-4d7fa48d21d4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ad7de632-aba6-40e0-a3e3-4d7fa48d21d4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:31.723591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:31.724554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:31.725513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:31.725513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:31.725513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:31.725513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:31.725513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:31.725513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:31.725513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:31.725513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:31.725513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 17:11:31.725513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:31.725513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 292/587] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:31.725513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7de632-aba6-40e0-a3e3-4d7fa48d21d4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:31.767919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938de2a0-a153-4277-9321-4cd44c82bce6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:31.802639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0b640-cb5d-4c5d-87db-75ebec63eff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:31.803250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0b640-cb5d-4c5d-87db-75ebec63eff4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:31.803527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0b640-cb5d-4c5d-87db-75ebec63eff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:31.803793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0b640-cb5d-4c5d-87db-75ebec63eff4 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:31.804576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0b640-cb5d-4c5d-87db-75ebec63eff4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:31.804882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0b640-cb5d-4c5d-87db-75ebec63eff4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:31.806320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0b640-cb5d-4c5d-87db-75ebec63eff4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:31.832868 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7de632-aba6-40e0-a3e3-4d7fa48d21d4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:31.847002 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 296/588] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Sep 27 17:11:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:31.847002 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0b640-cb5d-4c5d-87db-75ebec63eff4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:31.858578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:31.874526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b775291-5891-41c1-afe9-dc294a8e1f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:31.875138 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b775291-5891-41c1-afe9-dc294a8e1f71 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:31.875414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b775291-5891-41c1-afe9-dc294a8e1f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:31.875688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b775291-5891-41c1-afe9-dc294a8e1f71 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:31.883738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b775291-5891-41c1-afe9-dc294a8e1f71 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:31.885187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b775291-5891-41c1-afe9-dc294a8e1f71 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-8b775291-5891-41c1-afe9-dc294a8e1f71 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:31.897047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:31.898044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:31.903995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:31.903995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:31.903995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:31.903995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:31.903995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:31.903995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:31.903995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:31.903995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:31.903995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 27 17:11:31.903995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:31.903995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 293/589] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:31.903995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b775291-5891-41c1-afe9-dc294a8e1f71 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:31.916105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:31.948346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0b640-cb5d-4c5d-87db-75ebec63eff4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:31.983004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56ce44-97c0-4116-9474-13f2acdaa15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:31.983901 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56ce44-97c0-4116-9474-13f2acdaa15f None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:31.984175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56ce44-97c0-4116-9474-13f2acdaa15f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:31.984448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56ce44-97c0-4116-9474-13f2acdaa15f None admin] PATH_INFO: `/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/6cc89c1a7e684612a73ec9fef6bda780/roles/fafa3faaeb1344d2b6f639709d49e59b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:31.999680 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56ce44-97c0-4116-9474-13f2acdaa15f None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1f2574a4344711a56722bca143ae6c, user_id=6cc89c1a7e684612a73ec9fef6bda780, role_id=fafa3faaeb1344d2b6f639709d49e59b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:32.001139 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56ce44-97c0-4116-9474-13f2acdaa15f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:32.026108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 297/590] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:11:31 2022] PUT /identity/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/6cc89c1a7e684612a73ec9fef6bda780/roles/fafa3faaeb1344d2b6f639709d49e59b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:11:32.026108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56ce44-97c0-4116-9474-13f2acdaa15f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:32.030600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b775291-5891-41c1-afe9-dc294a8e1f71 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:32.068478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ea9e0-ae7b-46bc-9201-4c175af0d7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:32.069115 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ea9e0-ae7b-46bc-9201-4c175af0d7db None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:32.069392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ea9e0-ae7b-46bc-9201-4c175af0d7db None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:32.069665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ea9e0-ae7b-46bc-9201-4c175af0d7db None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:32.070447 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ea9e0-ae7b-46bc-9201-4c175af0d7db None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:32.070752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ea9e0-ae7b-46bc-9201-4c175af0d7db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:32.072322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ea9e0-ae7b-46bc-9201-4c175af0d7db None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:32.092742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 294/591] 10.222.0.55 () {56 vars in 1137 bytes} [Tue Sep 27 17:11:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:32.093633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ea9e0-ae7b-46bc-9201-4c175af0d7db None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:32.137132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:32.163089 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56ce44-97c0-4116-9474-13f2acdaa15f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:32.186793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:32.206246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff802f-8ee2-4ba4-a493-178dccc01ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:32.207091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff802f-8ee2-4ba4-a493-178dccc01ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:32.207285 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff802f-8ee2-4ba4-a493-178dccc01ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:32.207550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff802f-8ee2-4ba4-a493-178dccc01ac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:32.213944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ea9e0-ae7b-46bc-9201-4c175af0d7db None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:32.219549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff802f-8ee2-4ba4-a493-178dccc01ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:32.221583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff802f-8ee2-4ba4-a493-178dccc01ac7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-4aff802f-8ee2-4ba4-a493-178dccc01ac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:32.240963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:32.241934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:32.242872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:32.242872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:32.242872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:32.243041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 298/592] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:32.244202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff802f-8ee2-4ba4-a493-178dccc01ac7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:32.259786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35140138-2e39-49cb-8349-d61132b63357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:32.260395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35140138-2e39-49cb-8349-d61132b63357 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:32.260709 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35140138-2e39-49cb-8349-d61132b63357 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:32.260939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35140138-2e39-49cb-8349-d61132b63357 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:32.261688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35140138-2e39-49cb-8349-d61132b63357 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:32.261990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35140138-2e39-49cb-8349-d61132b63357 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:32.263457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35140138-2e39-49cb-8349-d61132b63357 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:32.282342 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 295/593] 10.222.0.55 () {56 vars in 1139 bytes} [Tue Sep 27 17:11:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:32.283517 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35140138-2e39-49cb-8349-d61132b63357 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:32.377586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff802f-8ee2-4ba4-a493-178dccc01ac7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:32.394286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:32.398603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35140138-2e39-49cb-8349-d61132b63357 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:32.424204 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce55c4-73bb-44a3-9ad5-87a4f95ee87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:32.424966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce55c4-73bb-44a3-9ad5-87a4f95ee87b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:32.425319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce55c4-73bb-44a3-9ad5-87a4f95ee87b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:32.425656 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce55c4-73bb-44a3-9ad5-87a4f95ee87b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:32.440950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce55c4-73bb-44a3-9ad5-87a4f95ee87b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:32.442594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce55c4-73bb-44a3-9ad5-87a4f95ee87b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:32.443187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbbb63-418d-4cef-9a2f-989c705fede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:32.444478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbbb63-418d-4cef-9a2f-989c705fede9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:32.444894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbbb63-418d-4cef-9a2f-989c705fede9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:32.445452 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbbb63-418d-4cef-9a2f-989c705fede9 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:32.446602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbbb63-418d-4cef-9a2f-989c705fede9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:32.447064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbbb63-418d-4cef-9a2f-989c705fede9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:32.448634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbbb63-418d-4cef-9a2f-989c705fede9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-0dce55c4-73bb-44a3-9ad5-87a4f95ee87b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:32.465241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:32.466293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:32.467212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:32.467212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:32.467212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:32.467457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 296/594] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:32.468525 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbbb63-418d-4cef-9a2f-989c705fede9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:32.472251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 299/595] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:32.474401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce55c4-73bb-44a3-9ad5-87a4f95ee87b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:32.512357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:32.598599 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbbb63-418d-4cef-9a2f-989c705fede9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:32.610112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce55c4-73bb-44a3-9ad5-87a4f95ee87b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:32.637954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e91c2-65a8-4af1-ae90-e2e196148aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:32.638699 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e91c2-65a8-4af1-ae90-e2e196148aeb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:32.638976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e91c2-65a8-4af1-ae90-e2e196148aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:32.639251 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e91c2-65a8-4af1-ae90-e2e196148aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:32.645711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cb8c4-91cf-4335-90b8-54c228ca9483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:32.646386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cb8c4-91cf-4335-90b8-54c228ca9483 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:32.646664 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cb8c4-91cf-4335-90b8-54c228ca9483 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:32.646932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cb8c4-91cf-4335-90b8-54c228ca9483 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:32.651427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e91c2-65a8-4af1-ae90-e2e196148aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:32.652912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696e91c2-65a8-4af1-ae90-e2e196148aeb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:32.653771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cb8c4-91cf-4335-90b8-54c228ca9483 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:32.655227 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cb8c4-91cf-4335-90b8-54c228ca9483 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-1d6cb8c4-91cf-4335-90b8-54c228ca9483 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:32.666829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:32.667707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:32.673532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:32.673532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:32.673532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:32.673532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:32.673532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:32.673532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:32.673532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:32.673532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:32.673532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 300/596] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-696e91c2-65a8-4af1-ae90-e2e196148aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:32.674011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:32.675197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:32.676157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:32.676157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:32.676157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:32.676328 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cb8c4-91cf-4335-90b8-54c228ca9483 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:32.676398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 297/597] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:32.676398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6ed82-0e4b-446e-b823-eca0aacc58f2 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:32.676398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6ed82-0e4b-446e-b823-eca0aacc58f2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:32.676398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6ed82-0e4b-446e-b823-eca0aacc58f2 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:32.677640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 298/598] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:32.678234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6ed82-0e4b-446e-b823-eca0aacc58f2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:32.690110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:32.763663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:32.781823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cb8c4-91cf-4335-90b8-54c228ca9483 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:32.790696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6ed82-0e4b-446e-b823-eca0aacc58f2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:32.818787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529ffad-4fcd-4d9b-9ede-02eac77d9536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:32.819538 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529ffad-4fcd-4d9b-9ede-02eac77d9536 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:32.819816 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529ffad-4fcd-4d9b-9ede-02eac77d9536 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:32.820086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529ffad-4fcd-4d9b-9ede-02eac77d9536 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:32.820732 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529ffad-4fcd-4d9b-9ede-02eac77d9536 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:32.821060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529ffad-4fcd-4d9b-9ede-02eac77d9536 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:32.822476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529ffad-4fcd-4d9b-9ede-02eac77d9536 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:32.826006 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da4b58-db45-4c0b-bfd2-1961003a19de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:32.826649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da4b58-db45-4c0b-bfd2-1961003a19de None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:32.826926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da4b58-db45-4c0b-bfd2-1961003a19de None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:32.827199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da4b58-db45-4c0b-bfd2-1961003a19de None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:32.827887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da4b58-db45-4c0b-bfd2-1961003a19de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:32.828193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da4b58-db45-4c0b-bfd2-1961003a19de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:32.829681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da4b58-db45-4c0b-bfd2-1961003a19de None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:32.835617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 301/599] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:32.836738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529ffad-4fcd-4d9b-9ede-02eac77d9536 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:32.848229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 299/600] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:32.849488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84296229-cd8f-4502-aa07-e03342e66ad1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:32.849765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84296229-cd8f-4502-aa07-e03342e66ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:32.850036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84296229-cd8f-4502-aa07-e03342e66ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:32.869988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-84296229-cd8f-4502-aa07-e03342e66ad1 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:32.871206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-84296229-cd8f-4502-aa07-e03342e66ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:32.936545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:32.954703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 300/601] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:32.955683 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84296229-cd8f-4502-aa07-e03342e66ad1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:32.958062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529ffad-4fcd-4d9b-9ede-02eac77d9536 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:33.000562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0802b-03b0-4368-bd28-6e37dc97f184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:33.001181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0802b-03b0-4368-bd28-6e37dc97f184 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:33.001497 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0802b-03b0-4368-bd28-6e37dc97f184 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:33.001770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0802b-03b0-4368-bd28-6e37dc97f184 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:33.002480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0802b-03b0-4368-bd28-6e37dc97f184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:33.002786 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0802b-03b0-4368-bd28-6e37dc97f184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:33.004189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0802b-03b0-4368-bd28-6e37dc97f184 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:33.016489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 302/602] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:33.021654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0802b-03b0-4368-bd28-6e37dc97f184 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:33.026032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:33.094843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84296229-cd8f-4502-aa07-e03342e66ad1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:33.130866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0802b-03b0-4368-bd28-6e37dc97f184 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:33.141684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc1352-620b-45c4-b006-64f2f6fd1eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:33.143090 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc1352-620b-45c4-b006-64f2f6fd1eee None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:33.143493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc1352-620b-45c4-b006-64f2f6fd1eee None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:33.143772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc1352-620b-45c4-b006-64f2f6fd1eee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:33.152806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc1352-620b-45c4-b006-64f2f6fd1eee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:33.154563 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc1352-620b-45c4-b006-64f2f6fd1eee None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:33.167326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41276cc-7d86-420e-be3d-7d36467c6d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:33.168088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41276cc-7d86-420e-be3d-7d36467c6d30 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:33.168517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41276cc-7d86-420e-be3d-7d36467c6d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:33.168889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41276cc-7d86-420e-be3d-7d36467c6d30 None admin] PATH_INFO: `/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/5537185f42474cfd86fca70fa8beddd2/roles/fafa3faaeb1344d2b6f639709d49e59b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-49dc1352-620b-45c4-b006-64f2f6fd1eee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:33.169978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:33.170904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:33.171806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:33.171806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:33.171806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:33.171806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:33.171806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:33.171806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:33.171806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:33.171806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:33.176157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 301/603] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:33.177926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc1352-620b-45c4-b006-64f2f6fd1eee None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:33.189516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41276cc-7d86-420e-be3d-7d36467c6d30 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1f2574a4344711a56722bca143ae6c, user_id=5537185f42474cfd86fca70fa8beddd2, role_id=fafa3faaeb1344d2b6f639709d49e59b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:33.191140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41276cc-7d86-420e-be3d-7d36467c6d30 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:33.210162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 303/604] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:11:33 2022] PUT /identity/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/5537185f42474cfd86fca70fa8beddd2/roles/fafa3faaeb1344d2b6f639709d49e59b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:11:33.211241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41276cc-7d86-420e-be3d-7d36467c6d30 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:33.267581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:33.294362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:33.344164 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc1352-620b-45c4-b006-64f2f6fd1eee None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:33.350398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41276cc-7d86-420e-be3d-7d36467c6d30 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:33.393185 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f6221-5f40-47e8-bf99-f65e4d11aa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:33.394103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f6221-5f40-47e8-bf99-f65e4d11aa61 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:33.394524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f6221-5f40-47e8-bf99-f65e4d11aa61 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:33.394935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f6221-5f40-47e8-bf99-f65e4d11aa61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:33.395214 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55024e93-e12d-4276-856b-280410410919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:33.395824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55024e93-e12d-4276-856b-280410410919 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:33.396096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55024e93-e12d-4276-856b-280410410919 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:33.396359 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55024e93-e12d-4276-856b-280410410919 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:33.397059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55024e93-e12d-4276-856b-280410410919 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:33.397358 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55024e93-e12d-4276-856b-280410410919 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:33.399044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55024e93-e12d-4276-856b-280410410919 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:33.404981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f6221-5f40-47e8-bf99-f65e4d11aa61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:33.406637 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f6221-5f40-47e8-bf99-f65e4d11aa61 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:33.414573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 304/605] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:33.415791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55024e93-e12d-4276-856b-280410410919 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-8f9f6221-5f40-47e8-bf99-f65e4d11aa61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:33.422051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:33.423165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:33.424361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:33.424361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:33.424361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:33.424361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:33.424361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:33.424361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:33.424361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:33.424361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:33.426838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 302/606] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:33.428242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f6221-5f40-47e8-bf99-f65e4d11aa61 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:33.538809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55024e93-e12d-4276-856b-280410410919 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:33.555063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f6221-5f40-47e8-bf99-f65e4d11aa61 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:33.557116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:33.576593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:33.585410 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905a840-3077-406a-8872-07eab668f7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:33.586517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905a840-3077-406a-8872-07eab668f7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:33.586944 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905a840-3077-406a-8872-07eab668f7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:33.587333 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905a840-3077-406a-8872-07eab668f7d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:33.597592 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ef6b8-7009-4b4d-883b-96aff7e5ef60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:33.598393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ef6b8-7009-4b4d-883b-96aff7e5ef60 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:33.598766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ef6b8-7009-4b4d-883b-96aff7e5ef60 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:33.599297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ef6b8-7009-4b4d-883b-96aff7e5ef60 None admin] PATH_INFO: `/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/528981e41f0b436b82989d62c713d993/roles/fafa3faaeb1344d2b6f639709d49e59b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:33.600006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905a840-3077-406a-8872-07eab668f7d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a905a840-3077-406a-8872-07eab668f7d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:33.606793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:33.608381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 305/607] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:33.609561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905a840-3077-406a-8872-07eab668f7d0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:33.619240 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ef6b8-7009-4b4d-883b-96aff7e5ef60 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1f2574a4344711a56722bca143ae6c, user_id=528981e41f0b436b82989d62c713d993, role_id=fafa3faaeb1344d2b6f639709d49e59b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:33.620921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ef6b8-7009-4b4d-883b-96aff7e5ef60 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:33.637555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 303/608] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:11:33 2022] PUT /identity/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/528981e41f0b436b82989d62c713d993/roles/fafa3faaeb1344d2b6f639709d49e59b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:11:33.645592 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ef6b8-7009-4b4d-883b-96aff7e5ef60 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:33.721960 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905a840-3077-406a-8872-07eab668f7d0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:33.760378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1c7cc-5cc6-49a9-bbc6-15916ca4bcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:33.761145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1c7cc-5cc6-49a9-bbc6-15916ca4bcc7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:33.761429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1c7cc-5cc6-49a9-bbc6-15916ca4bcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:33.761701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1c7cc-5cc6-49a9-bbc6-15916ca4bcc7 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:33.762514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1c7cc-5cc6-49a9-bbc6-15916ca4bcc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:33.762979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1c7cc-5cc6-49a9-bbc6-15916ca4bcc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:33.764515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1c7cc-5cc6-49a9-bbc6-15916ca4bcc7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:33.791790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 306/609] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:33.798627 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1c7cc-5cc6-49a9-bbc6-15916ca4bcc7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:33.871266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ef6b8-7009-4b4d-883b-96aff7e5ef60 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:33.884739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:33.918327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffe4ba-5741-4527-8ee2-22e6c2af831c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:33.919142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffe4ba-5741-4527-8ee2-22e6c2af831c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:33.919424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffe4ba-5741-4527-8ee2-22e6c2af831c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:33.919697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffe4ba-5741-4527-8ee2-22e6c2af831c None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:33.920372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffe4ba-5741-4527-8ee2-22e6c2af831c None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:33.920674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffe4ba-5741-4527-8ee2-22e6c2af831c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:33.922466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffe4ba-5741-4527-8ee2-22e6c2af831c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:33.936491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 304/610] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:33.943955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffe4ba-5741-4527-8ee2-22e6c2af831c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:33.984889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:33.991176 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1c7cc-5cc6-49a9-bbc6-15916ca4bcc7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:34.070043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffe4ba-5741-4527-8ee2-22e6c2af831c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:34.078821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc985a-934e-4dc1-83b8-610e64087256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:34.080104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc985a-934e-4dc1-83b8-610e64087256 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:34.080444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc985a-934e-4dc1-83b8-610e64087256 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:34.080727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc985a-934e-4dc1-83b8-610e64087256 None admin] PATH_INFO: `/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/5d906452199844149966c9c86db4684e/roles/fafa3faaeb1344d2b6f639709d49e59b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:34.113822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc985a-934e-4dc1-83b8-610e64087256 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1f2574a4344711a56722bca143ae6c, user_id=5d906452199844149966c9c86db4684e, role_id=fafa3faaeb1344d2b6f639709d49e59b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:34.116706 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc985a-934e-4dc1-83b8-610e64087256 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:34.125363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe050ae4-b3f1-42de-a425-5de32ebe38d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:34.127486 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe050ae4-b3f1-42de-a425-5de32ebe38d5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:34.128085 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe050ae4-b3f1-42de-a425-5de32ebe38d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:34.128675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe050ae4-b3f1-42de-a425-5de32ebe38d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:34.139311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 307/611] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:11:33 2022] PUT /identity/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/5d906452199844149966c9c86db4684e/roles/fafa3faaeb1344d2b6f639709d49e59b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:11:34.151541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe050ae4-b3f1-42de-a425-5de32ebe38d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:34.153121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe050ae4-b3f1-42de-a425-5de32ebe38d5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-fe050ae4-b3f1-42de-a425-5de32ebe38d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:34.173896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:34.175027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:34.175988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:34.175988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:34.175988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:34.175988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 305/612] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:34.181073 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc985a-934e-4dc1-83b8-610e64087256 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:34.305192 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:34.327374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc985a-934e-4dc1-83b8-610e64087256 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:34.358622 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d813d05-a2ff-4d8d-8a92-f6753e025a2e None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:34.358978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d813d05-a2ff-4d8d-8a92-f6753e025a2e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:34.359155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d813d05-a2ff-4d8d-8a92-f6753e025a2e None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:34.360922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 306/613] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:34.367490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b1358-15d0-4b64-97da-7478be4c0e7e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:34.367794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b1358-15d0-4b64-97da-7478be4c0e7e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:34.368026 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b1358-15d0-4b64-97da-7478be4c0e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:34.368478 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb0cbe-6074-4b36-9cf0-6601c5a54488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:34.369343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb0cbe-6074-4b36-9cf0-6601c5a54488 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:34.369835 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb0cbe-6074-4b36-9cf0-6601c5a54488 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:34.370342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb0cbe-6074-4b36-9cf0-6601c5a54488 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:34.371386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb0cbe-6074-4b36-9cf0-6601c5a54488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:34.371902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb0cbe-6074-4b36-9cf0-6601c5a54488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:34.373534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb0cbe-6074-4b36-9cf0-6601c5a54488 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:34.389007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 308/614] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:34.396016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb0cbe-6074-4b36-9cf0-6601c5a54488 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:34.400325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-022b1358-15d0-4b64-97da-7478be4c0e7e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:34.401498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-022b1358-15d0-4b64-97da-7478be4c0e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:34.484173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 307/615] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:34.525632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb0cbe-6074-4b36-9cf0-6601c5a54488 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:34.572666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c7daf-a06e-46c4-866e-8855dc310eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:34.573355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c7daf-a06e-46c4-866e-8855dc310eaf None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:34.573632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c7daf-a06e-46c4-866e-8855dc310eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:34.573895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c7daf-a06e-46c4-866e-8855dc310eaf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:34.583549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c7daf-a06e-46c4-866e-8855dc310eaf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:34.584984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c7daf-a06e-46c4-866e-8855dc310eaf None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-fe8c7daf-a06e-46c4-866e-8855dc310eaf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:34.597134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:34.598265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:34.599223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:34.599223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:34.599223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:34.599223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:34.599223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:34.599223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:34.599223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:34.599223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:34.599223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 27 17:11:34.599223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:34.599223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 309/616] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:34.604091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b1358-15d0-4b64-97da-7478be4c0e7e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:34.624828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:34.743183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b1358-15d0-4b64-97da-7478be4c0e7e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:34.782360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2497816-802a-4c5b-aefb-706bc49e6efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:34.783436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2497816-802a-4c5b-aefb-706bc49e6efb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:34.783945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2497816-802a-4c5b-aefb-706bc49e6efb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:34.784420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2497816-802a-4c5b-aefb-706bc49e6efb None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:34.785377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2497816-802a-4c5b-aefb-706bc49e6efb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:34.785888 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2497816-802a-4c5b-aefb-706bc49e6efb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:34.787551 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2497816-802a-4c5b-aefb-706bc49e6efb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:34.817710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 308/617] 10.222.0.55 () {56 vars in 1135 bytes} [Tue Sep 27 17:11:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:34.823002 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c7daf-a06e-46c4-866e-8855dc310eaf None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:34.830376 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:34.955448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c7daf-a06e-46c4-866e-8855dc310eaf None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:34.962978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2497816-802a-4c5b-aefb-706bc49e6efb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:34.998031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc741f0-9053-43fc-bab8-560ba3204d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:34.999376 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc741f0-9053-43fc-bab8-560ba3204d02 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:34.999846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc741f0-9053-43fc-bab8-560ba3204d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:35.000298 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc741f0-9053-43fc-bab8-560ba3204d02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:35.012164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc741f0-9053-43fc-bab8-560ba3204d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:35.014174 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc741f0-9053-43fc-bab8-560ba3204d02 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-0dc741f0-9053-43fc-bab8-560ba3204d02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:35.031367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:35.032308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:35.033280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:35.033280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:35.033280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:35.036052 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 310/618] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:35.046562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc741f0-9053-43fc-bab8-560ba3204d02 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:35.083652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:35.138760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2497816-802a-4c5b-aefb-706bc49e6efb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:35.156050 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:35.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd95d3-4fbb-4582-bfed-470fca61499e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:35.194297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd95d3-4fbb-4582-bfed-470fca61499e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:35.194793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd95d3-4fbb-4582-bfed-470fca61499e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:35.195769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd95d3-4fbb-4582-bfed-470fca61499e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:35.206936 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd95d3-4fbb-4582-bfed-470fca61499e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-cacd95d3-4fbb-4582-bfed-470fca61499e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:35.212012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:35.213315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:35.214403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:35.214403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:35.214403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:35.214403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:35.214403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:35.214403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:35.214403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:35.214403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:35.214403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:35.214403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:35.214403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:35.214403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:35.215225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc741f0-9053-43fc-bab8-560ba3204d02 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:35.221910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 309/619] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:35.223267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd95d3-4fbb-4582-bfed-470fca61499e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:35.279292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5b03e-962a-489d-b745-6b7eac62ac0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:35.279949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5b03e-962a-489d-b745-6b7eac62ac0f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:35.280233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5b03e-962a-489d-b745-6b7eac62ac0f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:35.280505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5b03e-962a-489d-b745-6b7eac62ac0f None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:35.281177 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5b03e-962a-489d-b745-6b7eac62ac0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:35.281451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5b03e-962a-489d-b745-6b7eac62ac0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:35.283495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5b03e-962a-489d-b745-6b7eac62ac0f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:35.308972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 311/620] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:35.315328 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5b03e-962a-489d-b745-6b7eac62ac0f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:35.332016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd95d3-4fbb-4582-bfed-470fca61499e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:35.363341 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:35.370250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc9e92-ab4a-446c-9517-a35c50a88b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:35.371142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9e92-ab4a-446c-9517-a35c50a88b38 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:35.371737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9e92-ab4a-446c-9517-a35c50a88b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:35.372161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9e92-ab4a-446c-9517-a35c50a88b38 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:35.372968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc9e92-ab4a-446c-9517-a35c50a88b38 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:35.373419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc9e92-ab4a-446c-9517-a35c50a88b38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:35.374973 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc9e92-ab4a-446c-9517-a35c50a88b38 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:35.389447 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 310/621] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:35.390969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc9e92-ab4a-446c-9517-a35c50a88b38 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:35.488573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5b03e-962a-489d-b745-6b7eac62ac0f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:35.526424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:35.549620 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc9e92-ab4a-446c-9517-a35c50a88b38 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:35.556401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1462fd-4903-4a4a-8043-da0bb3c2b114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:35.561985 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1462fd-4903-4a4a-8043-da0bb3c2b114 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:35.562301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1462fd-4903-4a4a-8043-da0bb3c2b114 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:35.562575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1462fd-4903-4a4a-8043-da0bb3c2b114 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:35.571170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1462fd-4903-4a4a-8043-da0bb3c2b114 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:35.572612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1462fd-4903-4a4a-8043-da0bb3c2b114 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ec1462fd-4903-4a4a-8043-da0bb3c2b114 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:35.593265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:35.594268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:35.595746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:35.595746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:35.595746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:35.595746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:35.595746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:35.595746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:35.595746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:35.595746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:35.602781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 312/622] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:35.608578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1462fd-4903-4a4a-8043-da0bb3c2b114 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:35.616588 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b9d7c-d8e2-4095-aec1-66ee8e52f245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:35.617219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b9d7c-d8e2-4095-aec1-66ee8e52f245 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:35.617497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b9d7c-d8e2-4095-aec1-66ee8e52f245 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:35.617764 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b9d7c-d8e2-4095-aec1-66ee8e52f245 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:35.643506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b9d7c-d8e2-4095-aec1-66ee8e52f245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:35.645821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b9d7c-d8e2-4095-aec1-66ee8e52f245 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ca8b9d7c-d8e2-4095-aec1-66ee8e52f245 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:35.677346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:35.678418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:35.679335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:35.679335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:35.679335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:35.681507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 311/623] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:35.687139 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b9d7c-d8e2-4095-aec1-66ee8e52f245 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:35.743248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1462fd-4903-4a4a-8043-da0bb3c2b114 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:35.775750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:35.812504 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4eedc2-56e0-47ec-bd1c-93cf37aacf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:35.813102 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eedc2-56e0-47ec-bd1c-93cf37aacf3e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:35.813379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eedc2-56e0-47ec-bd1c-93cf37aacf3e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:35.813693 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4eedc2-56e0-47ec-bd1c-93cf37aacf3e None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:35.814992 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eedc2-56e0-47ec-bd1c-93cf37aacf3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:35.815294 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eedc2-56e0-47ec-bd1c-93cf37aacf3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:35.816817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4eedc2-56e0-47ec-bd1c-93cf37aacf3e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:35.830180 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 313/624] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:35.831069 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab732154-b5ef-410f-8107-78b1d116e1ca None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:35.831347 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab732154-b5ef-410f-8107-78b1d116e1ca None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:35.831694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab732154-b5ef-410f-8107-78b1d116e1ca None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:35.836977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 314/625] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:35.837740 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab732154-b5ef-410f-8107-78b1d116e1ca None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:35.845658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b9d7c-d8e2-4095-aec1-66ee8e52f245 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:35.875535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:35.888717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58942472-1f28-420d-9ce1-452e386f1ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:35.889641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58942472-1f28-420d-9ce1-452e386f1ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:35.890169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58942472-1f28-420d-9ce1-452e386f1ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:35.890644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58942472-1f28-420d-9ce1-452e386f1ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:35.891561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58942472-1f28-420d-9ce1-452e386f1ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:35.892277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58942472-1f28-420d-9ce1-452e386f1ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:35.894173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58942472-1f28-420d-9ce1-452e386f1ed4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:35.908885 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 312/626] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:35.911109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71920817-7c9c-4a57-8d74-0fcf0a671c93 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:35.911679 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71920817-7c9c-4a57-8d74-0fcf0a671c93 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:35.912427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71920817-7c9c-4a57-8d74-0fcf0a671c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:35.931397 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-71920817-7c9c-4a57-8d74-0fcf0a671c93 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:35.932956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-71920817-7c9c-4a57-8d74-0fcf0a671c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:35.947335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab732154-b5ef-410f-8107-78b1d116e1ca None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:35.984625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3c9b1-19f7-4c9c-be19-0410e36cf63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:35.985345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3c9b1-19f7-4c9c-be19-0410e36cf63d None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:35.985658 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3c9b1-19f7-4c9c-be19-0410e36cf63d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:35.985943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3c9b1-19f7-4c9c-be19-0410e36cf63d None admin] PATH_INFO: `/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/39073006a0054b56bb1bba46d2d0768e/roles/fafa3faaeb1344d2b6f639709d49e59b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:36.002487 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3c9b1-19f7-4c9c-be19-0410e36cf63d None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1f2574a4344711a56722bca143ae6c, user_id=39073006a0054b56bb1bba46d2d0768e, role_id=fafa3faaeb1344d2b6f639709d49e59b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:36.004164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3c9b1-19f7-4c9c-be19-0410e36cf63d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:36.016096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 313/627] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:36.018251 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71920817-7c9c-4a57-8d74-0fcf0a671c93 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:36.035070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 315/628] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:11:35 2022] PUT /identity/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/39073006a0054b56bb1bba46d2d0768e/roles/fafa3faaeb1344d2b6f639709d49e59b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:11:36.061806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:36.133889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71920817-7c9c-4a57-8d74-0fcf0a671c93 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:36.169740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42c5f4-f2fb-4cb1-bedb-ecb787b95d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:36.170412 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c5f4-f2fb-4cb1-bedb-ecb787b95d11 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:36.170685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c5f4-f2fb-4cb1-bedb-ecb787b95d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:36.170951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c5f4-f2fb-4cb1-bedb-ecb787b95d11 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:36.182121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42c5f4-f2fb-4cb1-bedb-ecb787b95d11 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:36.182830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42c5f4-f2fb-4cb1-bedb-ecb787b95d11 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7f42c5f4-f2fb-4cb1-bedb-ecb787b95d11 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:36.196807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:36.197939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:36.198966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:36.198966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:36.198966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:36.198966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:36.198966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:36.198966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:36.198966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:36.198966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:36.198966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 27 17:11:36.198966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:36.198966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 314/629] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:36.204257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3c9b1-19f7-4c9c-be19-0410e36cf63d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:36.327946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42c5f4-f2fb-4cb1-bedb-ecb787b95d11 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:36.351636 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:36.380087 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3c9b1-19f7-4c9c-be19-0410e36cf63d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:36.424503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293333e-a059-4329-9aab-0c703a533b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:36.425118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293333e-a059-4329-9aab-0c703a533b6e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:36.425394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293333e-a059-4329-9aab-0c703a533b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:36.425645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293333e-a059-4329-9aab-0c703a533b6e None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:36.426635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293333e-a059-4329-9aab-0c703a533b6e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:36.426941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293333e-a059-4329-9aab-0c703a533b6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:36.428826 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c293333e-a059-4329-9aab-0c703a533b6e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:36.453045 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 316/630] 10.222.0.55 () {56 vars in 1137 bytes} [Tue Sep 27 17:11:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:36.459230 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293333e-a059-4329-9aab-0c703a533b6e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:36.468170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:36.478325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42c5f4-f2fb-4cb1-bedb-ecb787b95d11 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:36.522231 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a017cb-f379-4c29-9dde-e9897b798aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:36.523371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a017cb-f379-4c29-9dde-e9897b798aed None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:36.524125 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a017cb-f379-4c29-9dde-e9897b798aed None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:36.524622 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a017cb-f379-4c29-9dde-e9897b798aed None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:36.539237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a017cb-f379-4c29-9dde-e9897b798aed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-40a017cb-f379-4c29-9dde-e9897b798aed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:36.547624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:36.548731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:36.549578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:36.549578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:36.549578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:36.549578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:36.549578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:36.549578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:36.549578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:36.549578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:36.549578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:36.549578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:36.549578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:36.549578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:36.555514 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 315/631] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:36.564681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f560a-751f-4633-bd1d-65c6e8381ada None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:36.565406 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f560a-751f-4633-bd1d-65c6e8381ada None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:36.565884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f560a-751f-4633-bd1d-65c6e8381ada None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:36.568053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 316/632] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:11:36.569613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f560a-751f-4633-bd1d-65c6e8381ada None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:36.595581 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293333e-a059-4329-9aab-0c703a533b6e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:36.634124 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d1cc0-18bf-42c0-924c-9e7847c775d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:36.635093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d1cc0-18bf-42c0-924c-9e7847c775d1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:36.635688 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d1cc0-18bf-42c0-924c-9e7847c775d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:36.636158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d1cc0-18bf-42c0-924c-9e7847c775d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:36.653837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:36.690327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f560a-751f-4633-bd1d-65c6e8381ada None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:36.714552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d1cc0-18bf-42c0-924c-9e7847c775d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:36.716530 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d1cc0-18bf-42c0-924c-9e7847c775d1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:36.727110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c68404-72e3-4ab3-9a1f-fbe55bd167a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:36.727734 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c68404-72e3-4ab3-9a1f-fbe55bd167a6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:36.728012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c68404-72e3-4ab3-9a1f-fbe55bd167a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:36.728343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c68404-72e3-4ab3-9a1f-fbe55bd167a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:36.729029 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c68404-72e3-4ab3-9a1f-fbe55bd167a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:36.729385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c68404-72e3-4ab3-9a1f-fbe55bd167a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:36.731288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c68404-72e3-4ab3-9a1f-fbe55bd167a6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:36.746117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 317/633] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:36.746117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba252e-6467-43c4-92b9-8a7e875aed10 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:36.746117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba252e-6467-43c4-92b9-8a7e875aed10 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:36.746117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba252e-6467-43c4-92b9-8a7e875aed10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:36.765615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e9ba252e-6467-43c4-92b9-8a7e875aed10 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:36.767137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e9ba252e-6467-43c4-92b9-8a7e875aed10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:36.779152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:36.842438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 317/634] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:36.842739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 318/635] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:36.843685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba252e-6467-43c4-92b9-8a7e875aed10 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:36.853157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d1cc0-18bf-42c0-924c-9e7847c775d1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:36.907023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:36.957987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba252e-6467-43c4-92b9-8a7e875aed10 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:36.967079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d1cc0-18bf-42c0-924c-9e7847c775d1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:36.994710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a42227-5ceb-417f-90c4-3bb6b1d71960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:36.995340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a42227-5ceb-417f-90c4-3bb6b1d71960 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:36.995617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a42227-5ceb-417f-90c4-3bb6b1d71960 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:36.995888 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a42227-5ceb-417f-90c4-3bb6b1d71960 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:37.005822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a8070-1138-4475-bade-e437c09ca383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:37.006737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a8070-1138-4475-bade-e437c09ca383 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:37.007221 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a8070-1138-4475-bade-e437c09ca383 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:37.007469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a42227-5ceb-417f-90c4-3bb6b1d71960 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:37.007918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a8070-1138-4475-bade-e437c09ca383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:37.008954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a42227-5ceb-417f-90c4-3bb6b1d71960 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:37.016508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:37.020420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a8070-1138-4475-bade-e437c09ca383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:37.022308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a8070-1138-4475-bade-e437c09ca383 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-33a42227-5ceb-417f-90c4-3bb6b1d71960 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:37.025811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:37.026996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:37.028121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:37.028121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:37.028121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:37.034373 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 319/636] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:37.035700 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7415fa-588e-409b-8427-aa74dd87b196 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:37.036081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7415fa-588e-409b-8427-aa74dd87b196 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:37.036451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7415fa-588e-409b-8427-aa74dd87b196 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:37.038085 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 320/637] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:37.038922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7415fa-588e-409b-8427-aa74dd87b196 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-be1a8070-1138-4475-bade-e437c09ca383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:37.052155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:37.053297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:37.054268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:37.054268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:37.054268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:37.056387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 318/638] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:37.057799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab255c48-e09b-47bb-baf5-402bf32a18ae None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:37.058281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab255c48-e09b-47bb-baf5-402bf32a18ae None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:37.058750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab255c48-e09b-47bb-baf5-402bf32a18ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:37.078697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ab255c48-e09b-47bb-baf5-402bf32a18ae None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:37.080255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ab255c48-e09b-47bb-baf5-402bf32a18ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:37.151600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7415fa-588e-409b-8427-aa74dd87b196 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:37.161889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 319/639] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:37.162815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab255c48-e09b-47bb-baf5-402bf32a18ae None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:37.192454 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:37.202037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92a4e7-4a79-4ca9-9302-b09273a4ae4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:37.204003 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92a4e7-4a79-4ca9-9302-b09273a4ae4d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:37.204571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92a4e7-4a79-4ca9-9302-b09273a4ae4d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:37.205062 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92a4e7-4a79-4ca9-9302-b09273a4ae4d None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:37.205920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92a4e7-4a79-4ca9-9302-b09273a4ae4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:37.206477 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92a4e7-4a79-4ca9-9302-b09273a4ae4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:37.208066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e92a4e7-4a79-4ca9-9302-b09273a4ae4d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:37.227128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 321/640] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:37.228340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92a4e7-4a79-4ca9-9302-b09273a4ae4d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:37.286596 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab255c48-e09b-47bb-baf5-402bf32a18ae None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:37.322282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:37.337568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58236dc0-24c6-43b3-92e8-c84536be2ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:37.338213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58236dc0-24c6-43b3-92e8-c84536be2ffc None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:37.338495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58236dc0-24c6-43b3-92e8-c84536be2ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:37.338773 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58236dc0-24c6-43b3-92e8-c84536be2ffc None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:37.339420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58236dc0-24c6-43b3-92e8-c84536be2ffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:37.339725 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58236dc0-24c6-43b3-92e8-c84536be2ffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:37.341111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58236dc0-24c6-43b3-92e8-c84536be2ffc None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:37.357921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 320/641] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:37.359123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58236dc0-24c6-43b3-92e8-c84536be2ffc None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:37.369876 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92a4e7-4a79-4ca9-9302-b09273a4ae4d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:37.412409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de30be3-b8f4-4931-9dd8-353049488aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:37.412964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de30be3-b8f4-4931-9dd8-353049488aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:37.413247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de30be3-b8f4-4931-9dd8-353049488aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:37.413562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de30be3-b8f4-4931-9dd8-353049488aa6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:37.428642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de30be3-b8f4-4931-9dd8-353049488aa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7de30be3-b8f4-4931-9dd8-353049488aa6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:37.435852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:37.436790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:37.437634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:37.437634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:37.437634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:37.437634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:37.437634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:37.437634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:37.437634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:37.437634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:37.437634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:37.437634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:37.437634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:37.437634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:37.437634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 322/642] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:37.438419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de30be3-b8f4-4931-9dd8-353049488aa6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:37.465988 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:37.482427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58236dc0-24c6-43b3-92e8-c84536be2ffc None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:37.519352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a8f0d-1e27-4c7d-b7e6-ab35c5013b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:37.519980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a8f0d-1e27-4c7d-b7e6-ab35c5013b79 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:37.520257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a8f0d-1e27-4c7d-b7e6-ab35c5013b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:37.520527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a8f0d-1e27-4c7d-b7e6-ab35c5013b79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:37.529199 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a8f0d-1e27-4c7d-b7e6-ab35c5013b79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:37.530703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a8f0d-1e27-4c7d-b7e6-ab35c5013b79 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-f12a8f0d-1e27-4c7d-b7e6-ab35c5013b79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:37.542653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:37.543925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:37.545036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:37.545036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:37.545036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:37.545036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:37.545036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:37.545036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:37.545036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:37.545036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:37.545036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:11:37.545036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:37.545036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 321/643] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Sep 27 17:11:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:37.546469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a8f0d-1e27-4c7d-b7e6-ab35c5013b79 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:37.561657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de30be3-b8f4-4931-9dd8-353049488aa6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:37.583489 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:37.602290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a72eb-5cf3-412a-ba84-9e166278424f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:37.602922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a72eb-5cf3-412a-ba84-9e166278424f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:37.603198 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a72eb-5cf3-412a-ba84-9e166278424f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:37.603467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a72eb-5cf3-412a-ba84-9e166278424f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:37.611710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a72eb-5cf3-412a-ba84-9e166278424f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:37.613160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a72eb-5cf3-412a-ba84-9e166278424f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-af3a72eb-5cf3-412a-ba84-9e166278424f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:37.624535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:37.625625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:37.627241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:37.627241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:37.627241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:37.627241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:37.627241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:37.627241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:37.627241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:37.627241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:37.627241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 323/644] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:37.627241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a72eb-5cf3-412a-ba84-9e166278424f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:37.677469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a8f0d-1e27-4c7d-b7e6-ab35c5013b79 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:37.711336 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dc28b-65d5-4ea0-bb8f-6cff74ca1b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:37.711947 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dc28b-65d5-4ea0-bb8f-6cff74ca1b7e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:37.712208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dc28b-65d5-4ea0-bb8f-6cff74ca1b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:37.712474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dc28b-65d5-4ea0-bb8f-6cff74ca1b7e None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:37.713128 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dc28b-65d5-4ea0-bb8f-6cff74ca1b7e None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:37.713424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dc28b-65d5-4ea0-bb8f-6cff74ca1b7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:37.714797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dc28b-65d5-4ea0-bb8f-6cff74ca1b7e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:37.728296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 322/645] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:37.728296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dc28b-65d5-4ea0-bb8f-6cff74ca1b7e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:37.736132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:37.741468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a72eb-5cf3-412a-ba84-9e166278424f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:37.779798 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d92fa-c1f8-4b2a-a63d-7e81928a9867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:37.780408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d92fa-c1f8-4b2a-a63d-7e81928a9867 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:37.780682 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d92fa-c1f8-4b2a-a63d-7e81928a9867 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:37.781050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d92fa-c1f8-4b2a-a63d-7e81928a9867 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:37.792601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d92fa-c1f8-4b2a-a63d-7e81928a9867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-fa4d92fa-c1f8-4b2a-a63d-7e81928a9867 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:37.799244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:37.800222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:37.801063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:37.801063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:37.801063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:37.801063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:37.801063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:37.801063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:37.801063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:37.801063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:37.801063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:37.801063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:37.801063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:37.801063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:37.805001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 324/646] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:37.805814 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d92fa-c1f8-4b2a-a63d-7e81928a9867 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:37.830686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dc28b-65d5-4ea0-bb8f-6cff74ca1b7e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:37.847166 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:37.865495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60715d1b-319d-4609-bb26-20e25ccbba9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:37.866594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60715d1b-319d-4609-bb26-20e25ccbba9d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:37.867043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60715d1b-319d-4609-bb26-20e25ccbba9d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:37.867315 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60715d1b-319d-4609-bb26-20e25ccbba9d None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:37.868039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60715d1b-319d-4609-bb26-20e25ccbba9d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:37.868337 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60715d1b-319d-4609-bb26-20e25ccbba9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:37.869743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60715d1b-319d-4609-bb26-20e25ccbba9d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:37.899916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 323/647] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Sep 27 17:11:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:37.899916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60715d1b-319d-4609-bb26-20e25ccbba9d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:37.913109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d92fa-c1f8-4b2a-a63d-7e81928a9867 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:37.955016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9204c1-8c9f-4ce8-bd25-27f72433b667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:37.955552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9204c1-8c9f-4ce8-bd25-27f72433b667 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:37.955828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9204c1-8c9f-4ce8-bd25-27f72433b667 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:37.956101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9204c1-8c9f-4ce8-bd25-27f72433b667 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:37.956792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9204c1-8c9f-4ce8-bd25-27f72433b667 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:37.957106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9204c1-8c9f-4ce8-bd25-27f72433b667 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:37.958536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9204c1-8c9f-4ce8-bd25-27f72433b667 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:37.977093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 325/648] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:37.977093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9204c1-8c9f-4ce8-bd25-27f72433b667 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:37.977991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:38.007705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60715d1b-319d-4609-bb26-20e25ccbba9d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:38.043399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4aeb8-ec6d-49fa-a522-499683df96d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:38.044162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4aeb8-ec6d-49fa-a522-499683df96d8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:38.044441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4aeb8-ec6d-49fa-a522-499683df96d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:38.045060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4aeb8-ec6d-49fa-a522-499683df96d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:38.058161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4aeb8-ec6d-49fa-a522-499683df96d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:38.059614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4aeb8-ec6d-49fa-a522-499683df96d8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-28d4aeb8-ec6d-49fa-a522-499683df96d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:38.075729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:38.076708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:38.077614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:38.077614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:38.077614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:38.077614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 324/649] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:38.078581 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4aeb8-ec6d-49fa-a522-499683df96d8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:38.084253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:38.089388 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9204c1-8c9f-4ce8-bd25-27f72433b667 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:38.127456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903671b2-1d2a-4781-a6b9-1ea41d96655e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:38.128051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903671b2-1d2a-4781-a6b9-1ea41d96655e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:38.128326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903671b2-1d2a-4781-a6b9-1ea41d96655e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:38.128594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903671b2-1d2a-4781-a6b9-1ea41d96655e None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:38.129253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903671b2-1d2a-4781-a6b9-1ea41d96655e None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:38.129552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903671b2-1d2a-4781-a6b9-1ea41d96655e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:38.130929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903671b2-1d2a-4781-a6b9-1ea41d96655e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:38.144073 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 326/650] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:38.144073 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903671b2-1d2a-4781-a6b9-1ea41d96655e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:38.184231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4aeb8-ec6d-49fa-a522-499683df96d8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:38.201552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:38.219789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bdc00-2886-4528-95b2-c36fd84128e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:38.220840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bdc00-2886-4528-95b2-c36fd84128e8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:38.221117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bdc00-2886-4528-95b2-c36fd84128e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:38.221384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9bdc00-2886-4528-95b2-c36fd84128e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:38.256511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903671b2-1d2a-4781-a6b9-1ea41d96655e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:38.290398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bdc00-2886-4528-95b2-c36fd84128e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:38.291845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9bdc00-2886-4528-95b2-c36fd84128e8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:38.292698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcc1fd-75b8-474c-8e0d-7cf51bcc0d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:38.293355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcc1fd-75b8-474c-8e0d-7cf51bcc0d5c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:38.293916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcc1fd-75b8-474c-8e0d-7cf51bcc0d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:38.294276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcc1fd-75b8-474c-8e0d-7cf51bcc0d5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:38.295976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcc1fd-75b8-474c-8e0d-7cf51bcc0d5c None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=6cc89c1a7e684612a73ec9fef6bda780) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:38.296350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcc1fd-75b8-474c-8e0d-7cf51bcc0d5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:38.297812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcc1fd-75b8-474c-8e0d-7cf51bcc0d5c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:38.308011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 327/651] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:38 2022] GET /identity/v3/role_assignments?user.id=6cc89c1a7e684612a73ec9fef6bda780&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:38.308841 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcc1fd-75b8-474c-8e0d-7cf51bcc0d5c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:38.347526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:38.411500 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 325/652] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:38.412617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bdc00-2886-4528-95b2-c36fd84128e8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:38.432560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:38.454759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcc1fd-75b8-474c-8e0d-7cf51bcc0d5c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:38.506007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40600a-223a-4f79-8aa1-4299ee6067b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:38.506727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40600a-223a-4f79-8aa1-4299ee6067b5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:38.507125 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40600a-223a-4f79-8aa1-4299ee6067b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:38.507399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40600a-223a-4f79-8aa1-4299ee6067b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:38.508057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40600a-223a-4f79-8aa1-4299ee6067b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:38.508356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40600a-223a-4f79-8aa1-4299ee6067b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:38.509895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40600a-223a-4f79-8aa1-4299ee6067b5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:38.525996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 328/653] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:38.527088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40600a-223a-4f79-8aa1-4299ee6067b5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:38.535132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9bdc00-2886-4528-95b2-c36fd84128e8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:38.575397 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb7995-fc29-4d9c-b9e7-7e833c4543da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:38.575628 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb7995-fc29-4d9c-b9e7-7e833c4543da None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:38.575897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb7995-fc29-4d9c-b9e7-7e833c4543da None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:38.576163 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb7995-fc29-4d9c-b9e7-7e833c4543da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:38.589914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb7995-fc29-4d9c-b9e7-7e833c4543da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:38.591585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb7995-fc29-4d9c-b9e7-7e833c4543da None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-cfeb7995-fc29-4d9c-b9e7-7e833c4543da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:38.607846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:38.608900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:38.609806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:38.609806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:38.609806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:38.609806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 326/654] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:38.609806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb7995-fc29-4d9c-b9e7-7e833c4543da None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:38.631280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:38.648942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40600a-223a-4f79-8aa1-4299ee6067b5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:38.683090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:38.683626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0080a6e-ed2b-43ae-b476-8525e1f2ea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:38.684167 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0080a6e-ed2b-43ae-b476-8525e1f2ea5a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:38.684444 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0080a6e-ed2b-43ae-b476-8525e1f2ea5a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:38.684722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0080a6e-ed2b-43ae-b476-8525e1f2ea5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:38.695284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0080a6e-ed2b-43ae-b476-8525e1f2ea5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:38.696719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0080a6e-ed2b-43ae-b476-8525e1f2ea5a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-e0080a6e-ed2b-43ae-b476-8525e1f2ea5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:38.712369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:38.713504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:38.714673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:38.714673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:38.714673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:38.717966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 329/655] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:38.719127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031ce36-63ea-4357-9e21-44629a8f1a85 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:38.719413 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031ce36-63ea-4357-9e21-44629a8f1a85 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:38.719687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031ce36-63ea-4357-9e21-44629a8f1a85 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:38.721020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 330/656] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:38.721450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb7995-fc29-4d9c-b9e7-7e833c4543da None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:38.721710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031ce36-63ea-4357-9e21-44629a8f1a85 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:38.761340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca802ba-d543-4ac1-ac03-4d27170ec3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:38.761891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca802ba-d543-4ac1-ac03-4d27170ec3ec None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:38.762193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca802ba-d543-4ac1-ac03-4d27170ec3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:38.762463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca802ba-d543-4ac1-ac03-4d27170ec3ec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:38.770398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca802ba-d543-4ac1-ac03-4d27170ec3ec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:38.771843 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca802ba-d543-4ac1-ac03-4d27170ec3ec None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-7ca802ba-d543-4ac1-ac03-4d27170ec3ec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:38.784407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:38.785498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:38.786549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:38.786549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:38.786549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:38.786549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:38.786549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:38.786549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:38.786549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:38.786549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:38.786549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 17:11:38.786549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:38.786549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 327/657] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:38.786549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca802ba-d543-4ac1-ac03-4d27170ec3ec None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:38.839775 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031ce36-63ea-4357-9e21-44629a8f1a85 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:38.876980 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c9d38-f236-4218-97d7-4fc1d3c5776d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:38.877583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c9d38-f236-4218-97d7-4fc1d3c5776d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:38.877859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c9d38-f236-4218-97d7-4fc1d3c5776d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:38.878157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c9d38-f236-4218-97d7-4fc1d3c5776d None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:38.879220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c9d38-f236-4218-97d7-4fc1d3c5776d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:38.879527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c9d38-f236-4218-97d7-4fc1d3c5776d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:38.880889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c9d38-f236-4218-97d7-4fc1d3c5776d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:38.883193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:38.892678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca802ba-d543-4ac1-ac03-4d27170ec3ec None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:38.893955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 331/658] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:38.895229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a22e34-a8e7-4fc2-8baf-40c48527b619 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:38.895508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a22e34-a8e7-4fc2-8baf-40c48527b619 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:38.895780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a22e34-a8e7-4fc2-8baf-40c48527b619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:38.914677 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f5a22e34-a8e7-4fc2-8baf-40c48527b619 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:38.915969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f5a22e34-a8e7-4fc2-8baf-40c48527b619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:38.928657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:38.934532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9ede8-6984-444b-ad14-fc1e275d4e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:38.935147 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9ede8-6984-444b-ad14-fc1e275d4e06 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:38.935424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9ede8-6984-444b-ad14-fc1e275d4e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:38.935899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9ede8-6984-444b-ad14-fc1e275d4e06 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:38.936590 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9ede8-6984-444b-ad14-fc1e275d4e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:38.936898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9ede8-6984-444b-ad14-fc1e275d4e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:38.938402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f9ede8-6984-444b-ad14-fc1e275d4e06 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:38.952001 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 328/659] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:38.952001 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9ede8-6984-444b-ad14-fc1e275d4e06 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:39.006097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 332/660] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:39.006097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a22e34-a8e7-4fc2-8baf-40c48527b619 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:39.056625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9ede8-6984-444b-ad14-fc1e275d4e06 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:39.099035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119145d-b486-47fd-a31f-ad095b97a0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:39.099896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119145d-b486-47fd-a31f-ad095b97a0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:39.100178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119145d-b486-47fd-a31f-ad095b97a0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:39.100453 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7119145d-b486-47fd-a31f-ad095b97a0c2 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:39.101450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119145d-b486-47fd-a31f-ad095b97a0c2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:39.101756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119145d-b486-47fd-a31f-ad095b97a0c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:39.104053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7119145d-b486-47fd-a31f-ad095b97a0c2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:39.122777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a22e34-a8e7-4fc2-8baf-40c48527b619 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:39.127938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 329/661] 10.222.0.55 () {56 vars in 1137 bytes} [Tue Sep 27 17:11:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:39.128878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119145d-b486-47fd-a31f-ad095b97a0c2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:39.157521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:39.168426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b908-0a9a-4806-a220-5399da5b34c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:39.169251 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b908-0a9a-4806-a220-5399da5b34c4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:39.169596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b908-0a9a-4806-a220-5399da5b34c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:39.169967 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581b908-0a9a-4806-a220-5399da5b34c4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:39.178292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581b908-0a9a-4806-a220-5399da5b34c4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:39.179831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581b908-0a9a-4806-a220-5399da5b34c4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:39.190967 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d581b908-0a9a-4806-a220-5399da5b34c4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:39.193267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:39.194559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:39.195836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:39.195836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:39.195836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:39.195836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:39.195836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:39.195836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:39.195836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:39.195836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:39.195836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 27 17:11:39.195836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:39.200627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 333/662] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:39.201608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b908-0a9a-4806-a220-5399da5b34c4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:39.260537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7119145d-b486-47fd-a31f-ad095b97a0c2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:39.300488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e138e-5161-4f98-a5a7-a85d47c019d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:39.301150 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e138e-5161-4f98-a5a7-a85d47c019d9 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:39.301427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e138e-5161-4f98-a5a7-a85d47c019d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:39.301970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e138e-5161-4f98-a5a7-a85d47c019d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:39.308717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e138e-5161-4f98-a5a7-a85d47c019d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:39.310842 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e138e-5161-4f98-a5a7-a85d47c019d9 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:39.320172 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581b908-0a9a-4806-a220-5399da5b34c4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-f17e138e-5161-4f98-a5a7-a85d47c019d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:39.325954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:39.327215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:39.328162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:39.328162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:39.328162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:39.328162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:39.328162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:39.328162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:39.328162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:39.328162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:39.328162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 330/663] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:39.328660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e138e-5161-4f98-a5a7-a85d47c019d9 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:39.364451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f172213-a2d6-4e6c-8fa3-059dab2a95ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:39.365287 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f172213-a2d6-4e6c-8fa3-059dab2a95ea None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:39.365783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f172213-a2d6-4e6c-8fa3-059dab2a95ea None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:39.366235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f172213-a2d6-4e6c-8fa3-059dab2a95ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:39.425938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:39.436608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e138e-5161-4f98-a5a7-a85d47c019d9 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:39.440356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f172213-a2d6-4e6c-8fa3-059dab2a95ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:39.441829 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f172213-a2d6-4e6c-8fa3-059dab2a95ea None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:39.446030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:39.473182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427a8b7-db98-4f50-8e06-dca6871b76cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:39.473762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427a8b7-db98-4f50-8e06-dca6871b76cc None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:39.474114 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427a8b7-db98-4f50-8e06-dca6871b76cc None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:39.474380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427a8b7-db98-4f50-8e06-dca6871b76cc None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:39.475033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427a8b7-db98-4f50-8e06-dca6871b76cc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:39.475334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427a8b7-db98-4f50-8e06-dca6871b76cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:39.476739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427a8b7-db98-4f50-8e06-dca6871b76cc None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:39.498290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 331/664] 10.222.0.55 () {56 vars in 1139 bytes} [Tue Sep 27 17:11:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:39.498290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427a8b7-db98-4f50-8e06-dca6871b76cc None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:39.551945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 334/665] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:39.552799 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f172213-a2d6-4e6c-8fa3-059dab2a95ea None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:39.603690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427a8b7-db98-4f50-8e06-dca6871b76cc None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:39.641466 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34df20-2505-4a5d-9e4d-685a9401370b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:39.642186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34df20-2505-4a5d-9e4d-685a9401370b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:39.642462 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34df20-2505-4a5d-9e4d-685a9401370b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:39.642731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34df20-2505-4a5d-9e4d-685a9401370b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:39.645493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:39.653631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34df20-2505-4a5d-9e4d-685a9401370b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-4c34df20-2505-4a5d-9e4d-685a9401370b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:39.658921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:39.660080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:39.661131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:39.661131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:39.661131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:39.661131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:39.661131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:39.661131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:39.661131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:39.661131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:39.661131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:39.661131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:39.661131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:39.661131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:39.664658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f172213-a2d6-4e6c-8fa3-059dab2a95ea None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:39.664895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 332/666] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:39.665714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34df20-2505-4a5d-9e4d-685a9401370b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:39.694691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:39.704840 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fbbb1-a594-456f-943f-b1325451f7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:39.705450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fbbb1-a594-456f-943f-b1325451f7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:39.705730 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fbbb1-a594-456f-943f-b1325451f7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:39.706003 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9fbbb1-a594-456f-943f-b1325451f7a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:39.706729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fbbb1-a594-456f-943f-b1325451f7a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:39.707030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fbbb1-a594-456f-943f-b1325451f7a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:39.708446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9fbbb1-a594-456f-943f-b1325451f7a6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:39.722468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 335/667] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:39.722468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fbbb1-a594-456f-943f-b1325451f7a6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:39.772792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34df20-2505-4a5d-9e4d-685a9401370b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:39.814344 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e499d2-d780-4866-a9b8-54475c1a10e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:39.815272 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e499d2-d780-4866-a9b8-54475c1a10e1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:39.815657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e499d2-d780-4866-a9b8-54475c1a10e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:39.815936 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e499d2-d780-4866-a9b8-54475c1a10e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:39.851036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9fbbb1-a594-456f-943f-b1325451f7a6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:39.884166 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e499d2-d780-4866-a9b8-54475c1a10e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:39.885536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e499d2-d780-4866-a9b8-54475c1a10e1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:39.890505 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcc066-79a4-4341-852f-fe278ddb6f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:39.891127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcc066-79a4-4341-852f-fe278ddb6f0b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:39.891404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcc066-79a4-4341-852f-fe278ddb6f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:39.892227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcc066-79a4-4341-852f-fe278ddb6f0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:39.903796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fcc066-79a4-4341-852f-fe278ddb6f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:39.905071 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fcc066-79a4-4341-852f-fe278ddb6f0b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:39.912881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:39.913639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-85fcc066-79a4-4341-852f-fe278ddb6f0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:39.923052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:39.924195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:39.925360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:39.925360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:39.925360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:39.931479 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 336/668] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:39.931479 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcc066-79a4-4341-852f-fe278ddb6f0b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:39.990703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 333/669] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:39.991938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e499d2-d780-4866-a9b8-54475c1a10e1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:40.043048 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcc066-79a4-4341-852f-fe278ddb6f0b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:40.081989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc5c6c-35b6-4ee5-89db-086d6f44a1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:40.082629 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc5c6c-35b6-4ee5-89db-086d6f44a1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:40.082995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc5c6c-35b6-4ee5-89db-086d6f44a1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:40.083169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc5c6c-35b6-4ee5-89db-086d6f44a1e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:40.083994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc5c6c-35b6-4ee5-89db-086d6f44a1e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:40.084102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc5c6c-35b6-4ee5-89db-086d6f44a1e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:40.085469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc5c6c-35b6-4ee5-89db-086d6f44a1e1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:40.098224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 337/670] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:40.099087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc5c6c-35b6-4ee5-89db-086d6f44a1e1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:40.104862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e499d2-d780-4866-a9b8-54475c1a10e1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:40.135097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:40.148169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b117e0-a3c0-4032-98bc-aea2a63351b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:40.148787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b117e0-a3c0-4032-98bc-aea2a63351b7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:40.149522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b117e0-a3c0-4032-98bc-aea2a63351b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:40.149794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b117e0-a3c0-4032-98bc-aea2a63351b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:40.150913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b117e0-a3c0-4032-98bc-aea2a63351b7 None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=5537185f42474cfd86fca70fa8beddd2) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:40.151229 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b117e0-a3c0-4032-98bc-aea2a63351b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:40.153566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b117e0-a3c0-4032-98bc-aea2a63351b7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:40.162376 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 334/671] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:39 2022] GET /identity/v3/role_assignments?user.id=5537185f42474cfd86fca70fa8beddd2&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:40.163449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b117e0-a3c0-4032-98bc-aea2a63351b7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:40.164524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:40.215409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc5c6c-35b6-4ee5-89db-086d6f44a1e1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:40.281782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea0d86-6121-4a95-8774-f347ef0bbf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:40.283224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea0d86-6121-4a95-8774-f347ef0bbf66 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:40.283649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea0d86-6121-4a95-8774-f347ef0bbf66 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:40.284090 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea0d86-6121-4a95-8774-f347ef0bbf66 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:40.284899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea0d86-6121-4a95-8774-f347ef0bbf66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:40.285362 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea0d86-6121-4a95-8774-f347ef0bbf66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:40.286920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea0d86-6121-4a95-8774-f347ef0bbf66 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:40.305721 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 338/672] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:40.306938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea0d86-6121-4a95-8774-f347ef0bbf66 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:40.317533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b117e0-a3c0-4032-98bc-aea2a63351b7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:40.370906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa6139-45fe-4b40-86f8-bbf8fd62f545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:40.371565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa6139-45fe-4b40-86f8-bbf8fd62f545 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:40.371844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa6139-45fe-4b40-86f8-bbf8fd62f545 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:40.372115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa6139-45fe-4b40-86f8-bbf8fd62f545 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:40.401118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa6139-45fe-4b40-86f8-bbf8fd62f545 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:40.402778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa6139-45fe-4b40-86f8-bbf8fd62f545 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-d5aa6139-45fe-4b40-86f8-bbf8fd62f545 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:40.421392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:40.422554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:40.423568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:40.423568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:40.423568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:40.429142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 335/673] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:40.429955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea0d86-6121-4a95-8774-f347ef0bbf66 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:40.430097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa6139-45fe-4b40-86f8-bbf8fd62f545 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:40.464203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:40.492353 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d7a12-2222-491c-b1c5-e48814a58fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:40.493443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d7a12-2222-491c-b1c5-e48814a58fca None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:40.493936 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d7a12-2222-491c-b1c5-e48814a58fca None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:40.494445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d7a12-2222-491c-b1c5-e48814a58fca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:40.497900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:40.508086 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d7a12-2222-491c-b1c5-e48814a58fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:40.509610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d7a12-2222-491c-b1c5-e48814a58fca None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-377d7a12-2222-491c-b1c5-e48814a58fca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:40.534762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:40.536182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:40.537136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:40.537136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:40.537136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:40.538630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 339/674] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:40.539528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d7a12-2222-491c-b1c5-e48814a58fca None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:40.595792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa6139-45fe-4b40-86f8-bbf8fd62f545 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:40.652078 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c047e70-b98e-4c96-a495-63b0fe74b13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:40.652646 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c047e70-b98e-4c96-a495-63b0fe74b13f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:40.652925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c047e70-b98e-4c96-a495-63b0fe74b13f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:40.653199 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c047e70-b98e-4c96-a495-63b0fe74b13f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:40.665831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c047e70-b98e-4c96-a495-63b0fe74b13f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:40.667329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c047e70-b98e-4c96-a495-63b0fe74b13f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-5c047e70-b98e-4c96-a495-63b0fe74b13f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:40.679846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:40.680801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:40.681709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:40.681709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:40.681709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:40.681709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:40.681709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:40.681709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:40.681709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:40.681709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:40.681709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 336/675] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:40.681709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c047e70-b98e-4c96-a495-63b0fe74b13f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:40.691941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d7a12-2222-491c-b1c5-e48814a58fca None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:40.726354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b4167-6c5d-499e-981f-c53a5580bfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:40.727006 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b4167-6c5d-499e-981f-c53a5580bfa8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:40.727286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b4167-6c5d-499e-981f-c53a5580bfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:40.727558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b4167-6c5d-499e-981f-c53a5580bfa8 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:40.728198 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b4167-6c5d-499e-981f-c53a5580bfa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:40.728498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b4167-6c5d-499e-981f-c53a5580bfa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:40.729840 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b4167-6c5d-499e-981f-c53a5580bfa8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:40.744792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 340/676] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:40.745850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdba27a-9cd4-46f4-93fb-ab4a84be9d30 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:40.746160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdba27a-9cd4-46f4-93fb-ab4a84be9d30 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:40.746434 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdba27a-9cd4-46f4-93fb-ab4a84be9d30 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:40.747797 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 341/677] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:40.748393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdba27a-9cd4-46f4-93fb-ab4a84be9d30 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:40.780356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:40.804313 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c047e70-b98e-4c96-a495-63b0fe74b13f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:40.805860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:40.844252 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3a4d0-42a6-4bd9-ad78-aac618f046a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:40.845125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3a4d0-42a6-4bd9-ad78-aac618f046a5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:40.845401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3a4d0-42a6-4bd9-ad78-aac618f046a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:40.846099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3a4d0-42a6-4bd9-ad78-aac618f046a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:40.847002 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3a4d0-42a6-4bd9-ad78-aac618f046a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:40.847304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3a4d0-42a6-4bd9-ad78-aac618f046a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:40.849331 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d3a4d0-42a6-4bd9-ad78-aac618f046a5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:40.855203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdba27a-9cd4-46f4-93fb-ab4a84be9d30 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:40.863140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 337/678] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:40.863975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3a4d0-42a6-4bd9-ad78-aac618f046a5 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:40.897281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdb6ae-00eb-4b54-958d-a4bd0d9c2755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:40.898041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdb6ae-00eb-4b54-958d-a4bd0d9c2755 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:40.898441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdb6ae-00eb-4b54-958d-a4bd0d9c2755 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:40.898810 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdb6ae-00eb-4b54-958d-a4bd0d9c2755 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:40.899596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdb6ae-00eb-4b54-958d-a4bd0d9c2755 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:40.900018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdb6ae-00eb-4b54-958d-a4bd0d9c2755 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:40.901535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdb6ae-00eb-4b54-958d-a4bd0d9c2755 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:40.914184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 342/679] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:40.915456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306653f8-5cd1-4f73-8faf-4a1d29763cc3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:40.915836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306653f8-5cd1-4f73-8faf-4a1d29763cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:40.916206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306653f8-5cd1-4f73-8faf-4a1d29763cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:40.935280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-306653f8-5cd1-4f73-8faf-4a1d29763cc3 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:40.936642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-306653f8-5cd1-4f73-8faf-4a1d29763cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:40.976499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3a4d0-42a6-4bd9-ad78-aac618f046a5 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:41.007795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 343/680] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:41.008685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306653f8-5cd1-4f73-8faf-4a1d29763cc3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:41.021321 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a77199-d1bd-4c87-b743-bc112781174b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:41.022287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a77199-d1bd-4c87-b743-bc112781174b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:41.022775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a77199-d1bd-4c87-b743-bc112781174b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:41.023245 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a77199-d1bd-4c87-b743-bc112781174b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:41.030692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a77199-d1bd-4c87-b743-bc112781174b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:41.032982 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a77199-d1bd-4c87-b743-bc112781174b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:41.033607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-41a77199-d1bd-4c87-b743-bc112781174b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:41.045890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:41.046828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:41.047804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:41.047804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:41.047804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:41.047804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:41.047804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:41.047804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:41.047804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:41.047804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:41.054694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 338/681] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:41.055808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a77199-d1bd-4c87-b743-bc112781174b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:41.098427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:41.120023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306653f8-5cd1-4f73-8faf-4a1d29763cc3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:41.159069 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a77199-d1bd-4c87-b743-bc112781174b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:41.159325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6a67b-1787-4192-b008-699009cc3ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:41.159742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a67b-1787-4192-b008-699009cc3ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:41.160016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a67b-1787-4192-b008-699009cc3ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:41.160282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd6a67b-1787-4192-b008-699009cc3ff5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:41.167681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6a67b-1787-4192-b008-699009cc3ff5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:41.169047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd6a67b-1787-4192-b008-699009cc3ff5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-bfd6a67b-1787-4192-b008-699009cc3ff5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:41.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:41.185173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:41.186646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:41.186646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:41.186646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:41.186646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:41.186646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:41.186646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:41.186646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:41.186646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:41.186646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 27 17:11:41.186646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:41.190566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 344/682] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:41.191411 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6a67b-1787-4192-b008-699009cc3ff5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:41.200881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deac16-b6d9-4a64-835a-f8e30c6cd44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:41.201758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deac16-b6d9-4a64-835a-f8e30c6cd44e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:41.202322 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deac16-b6d9-4a64-835a-f8e30c6cd44e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:41.202805 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deac16-b6d9-4a64-835a-f8e30c6cd44e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:41.203783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10deac16-b6d9-4a64-835a-f8e30c6cd44e None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=528981e41f0b436b82989d62c713d993) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:41.204389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10deac16-b6d9-4a64-835a-f8e30c6cd44e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:41.206062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10deac16-b6d9-4a64-835a-f8e30c6cd44e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:41.218262 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 339/683] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:41 2022] GET /identity/v3/role_assignments?user.id=528981e41f0b436b82989d62c713d993&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:41.219566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deac16-b6d9-4a64-835a-f8e30c6cd44e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:41.321080 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:41.346094 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd6a67b-1787-4192-b008-699009cc3ff5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:41.366566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:41.385762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deac16-b6d9-4a64-835a-f8e30c6cd44e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:41.390640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6c8b4-8e07-4ba4-8ab0-e87c4ca4aef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:41.391255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6c8b4-8e07-4ba4-8ab0-e87c4ca4aef1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:41.392048 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6c8b4-8e07-4ba4-8ab0-e87c4ca4aef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:41.392324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6c8b4-8e07-4ba4-8ab0-e87c4ca4aef1 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:41.393026 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6c8b4-8e07-4ba4-8ab0-e87c4ca4aef1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:41.393324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6c8b4-8e07-4ba4-8ab0-e87c4ca4aef1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:41.394799 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6c8b4-8e07-4ba4-8ab0-e87c4ca4aef1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:41.411940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 345/684] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:41.412948 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6c8b4-8e07-4ba4-8ab0-e87c4ca4aef1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:41.428203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae10137-8d18-40f1-b1cc-cc06da343549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:41.429037 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae10137-8d18-40f1-b1cc-cc06da343549 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:41.429470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae10137-8d18-40f1-b1cc-cc06da343549 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:41.429879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae10137-8d18-40f1-b1cc-cc06da343549 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:41.430792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae10137-8d18-40f1-b1cc-cc06da343549 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:41.431236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae10137-8d18-40f1-b1cc-cc06da343549 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:41.432791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae10137-8d18-40f1-b1cc-cc06da343549 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:41.453115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 340/685] 10.222.0.55 () {56 vars in 1135 bytes} [Tue Sep 27 17:11:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:41.454878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae10137-8d18-40f1-b1cc-cc06da343549 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:41.537237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6c8b4-8e07-4ba4-8ab0-e87c4ca4aef1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:41.571514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae10137-8d18-40f1-b1cc-cc06da343549 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:41.582554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06bf77a-8a02-475d-8903-d78c2951a07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:41.583495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bf77a-8a02-475d-8903-d78c2951a07f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:41.583915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bf77a-8a02-475d-8903-d78c2951a07f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:41.584286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bf77a-8a02-475d-8903-d78c2951a07f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:41.597170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06bf77a-8a02-475d-8903-d78c2951a07f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:41.599021 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06bf77a-8a02-475d-8903-d78c2951a07f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:41.601608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:41.613771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a7218-606f-4e96-a2c9-86451151f216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:41.614605 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a7218-606f-4e96-a2c9-86451151f216 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:41.615047 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a7218-606f-4e96-a2c9-86451151f216 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:41.615465 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a7218-606f-4e96-a2c9-86451151f216 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:41.616340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a7218-606f-4e96-a2c9-86451151f216 None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=5d906452199844149966c9c86db4684e) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:41.616790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a7218-606f-4e96-a2c9-86451151f216 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-a06bf77a-8a02-475d-8903-d78c2951a07f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:41.618324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:41.619379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:41.625045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:41.625045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:41.625045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:41.625045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 346/686] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:41.625045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06bf77a-8a02-475d-8903-d78c2951a07f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:41.625616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a7218-606f-4e96-a2c9-86451151f216 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:41.632969 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 341/687] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:41 2022] GET /identity/v3/role_assignments?user.id=5d906452199844149966c9c86db4684e&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:41.634180 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a7218-606f-4e96-a2c9-86451151f216 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:41.659404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:41.739501 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06bf77a-8a02-475d-8903-d78c2951a07f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:41.786848 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d8888-711c-42b9-985e-927d9506b30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:41.787474 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d8888-711c-42b9-985e-927d9506b30b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:41.787747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d8888-711c-42b9-985e-927d9506b30b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:41.788016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d8888-711c-42b9-985e-927d9506b30b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:41.846548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a7218-606f-4e96-a2c9-86451151f216 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:41.876950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:41.880628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d8888-711c-42b9-985e-927d9506b30b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:41.881894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d8888-711c-42b9-985e-927d9506b30b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:41.928617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5e416-2a2a-4eb8-a800-21a786b51592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:41.929374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5e416-2a2a-4eb8-a800-21a786b51592 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:41.929650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5e416-2a2a-4eb8-a800-21a786b51592 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:41.929912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5e416-2a2a-4eb8-a800-21a786b51592 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:41.930987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5e416-2a2a-4eb8-a800-21a786b51592 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:41.930987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5e416-2a2a-4eb8-a800-21a786b51592 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:41.932227 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5e416-2a2a-4eb8-a800-21a786b51592 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:41.948039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 342/688] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:41.953261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5e416-2a2a-4eb8-a800-21a786b51592 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:42.016046 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 347/689] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:42.022018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d8888-711c-42b9-985e-927d9506b30b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:42.058477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:42.085014 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5e416-2a2a-4eb8-a800-21a786b51592 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:42.121269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbeba20-b47a-422a-bd2a-e1775b608e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:42.121938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbeba20-b47a-422a-bd2a-e1775b608e43 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:42.122258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbeba20-b47a-422a-bd2a-e1775b608e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:42.122538 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbeba20-b47a-422a-bd2a-e1775b608e43 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:42.123194 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbeba20-b47a-422a-bd2a-e1775b608e43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:42.125125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbeba20-b47a-422a-bd2a-e1775b608e43 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:42.140903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:42.148313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d8888-711c-42b9-985e-927d9506b30b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:42.157847 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 343/690] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:11:41 2022] POST /identity/v3/users => generated 254 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:11:42.187332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d543f31-16a9-40df-b5ac-5d41df080d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:42.187965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d543f31-16a9-40df-b5ac-5d41df080d41 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:42.188240 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d543f31-16a9-40df-b5ac-5d41df080d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:42.188617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d543f31-16a9-40df-b5ac-5d41df080d41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:42.200445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d543f31-16a9-40df-b5ac-5d41df080d41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:42.201891 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d543f31-16a9-40df-b5ac-5d41df080d41 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-5d543f31-16a9-40df-b5ac-5d41df080d41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:42.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:42.219394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:42.220386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:42.220386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:42.220386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:42.220386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 348/691] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:42.225357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbeba20-b47a-422a-bd2a-e1775b608e43 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:42.397367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbeba20-b47a-422a-bd2a-e1775b608e43 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:42.423999 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:42.437567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b99242-3e37-49d9-a24f-156b595a240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:42.438260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b99242-3e37-49d9-a24f-156b595a240c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:42.439088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b99242-3e37-49d9-a24f-156b595a240c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:42.439364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b99242-3e37-49d9-a24f-156b595a240c None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:42.440065 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b99242-3e37-49d9-a24f-156b595a240c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:42.440371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b99242-3e37-49d9-a24f-156b595a240c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:42.441754 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b99242-3e37-49d9-a24f-156b595a240c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:42.454451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 344/692] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:42.459709 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d543f31-16a9-40df-b5ac-5d41df080d41 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:42.589684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d543f31-16a9-40df-b5ac-5d41df080d41 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:42.628816 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f4bbf-d989-416c-b7f0-4144036e88c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:42.629447 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f4bbf-d989-416c-b7f0-4144036e88c0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:42.629722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f4bbf-d989-416c-b7f0-4144036e88c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:42.629992 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f4bbf-d989-416c-b7f0-4144036e88c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:42.637222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f4bbf-d989-416c-b7f0-4144036e88c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:42.638684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f4bbf-d989-416c-b7f0-4144036e88c0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-725f4bbf-d989-416c-b7f0-4144036e88c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:42.662352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:42.663327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:42.664272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:42.664272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:42.664272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:42.664272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:42.664272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:42.664272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:42.664272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:42.664272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:42.664706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 349/693] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:42.671189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b99242-3e37-49d9-a24f-156b595a240c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:42.840213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b99242-3e37-49d9-a24f-156b595a240c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:42.842367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697c05b-5558-4a4e-90ac-e9c079eea148 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:42.842648 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697c05b-5558-4a4e-90ac-e9c079eea148 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:42.842918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697c05b-5558-4a4e-90ac-e9c079eea148 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:42.844515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 350/694] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:11:42.850331 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15971951-49c3-40d3-93f0-1b8136e41537 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:42.850612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15971951-49c3-40d3-93f0-1b8136e41537 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:42.850884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15971951-49c3-40d3-93f0-1b8136e41537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:42.866437 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:42.875954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-15971951-49c3-40d3-93f0-1b8136e41537 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:42.879818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-15971951-49c3-40d3-93f0-1b8136e41537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:42.911684 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2969562-327a-4e01-993c-30ce5beb40c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:42.912368 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2969562-327a-4e01-993c-30ce5beb40c2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:42.912619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2969562-327a-4e01-993c-30ce5beb40c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:42.912891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2969562-327a-4e01-993c-30ce5beb40c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:42.913595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2969562-327a-4e01-993c-30ce5beb40c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:42.913900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2969562-327a-4e01-993c-30ce5beb40c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:42.915434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2969562-327a-4e01-993c-30ce5beb40c2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:42.939793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 345/695] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:42.946174 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2969562-327a-4e01-993c-30ce5beb40c2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:42.995717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 351/696] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:11:43.021515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:43.097111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2969562-327a-4e01-993c-30ce5beb40c2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:43.131351 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebc831-f2ad-4419-ad55-ff517bf0a28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:43.131993 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebc831-f2ad-4419-ad55-ff517bf0a28a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:43.132274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebc831-f2ad-4419-ad55-ff517bf0a28a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:43.132544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebc831-f2ad-4419-ad55-ff517bf0a28a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:43.133266 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebc831-f2ad-4419-ad55-ff517bf0a28a None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=39073006a0054b56bb1bba46d2d0768e) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:43.133577 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebc831-f2ad-4419-ad55-ff517bf0a28a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:43.135046 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebc831-f2ad-4419-ad55-ff517bf0a28a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:43.141939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 346/697] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:42 2022] GET /identity/v3/role_assignments?user.id=39073006a0054b56bb1bba46d2d0768e&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:43.307608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15971951-49c3-40d3-93f0-1b8136e41537 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:43.429643 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15971951-49c3-40d3-93f0-1b8136e41537 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:43.466420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c20f01-6f8f-41af-a110-06bebcce89dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:43.467053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c20f01-6f8f-41af-a110-06bebcce89dc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:43.467389 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c20f01-6f8f-41af-a110-06bebcce89dc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:43.467668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c20f01-6f8f-41af-a110-06bebcce89dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:43.471158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:43.481102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c20f01-6f8f-41af-a110-06bebcce89dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-14c20f01-6f8f-41af-a110-06bebcce89dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:43.485929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:43.488043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:43.490010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:43.490010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:43.490010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:43.490010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:43.490010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:43.490010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:43.490010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:43.490010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:43.490010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:43.490010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:43.490010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:43.490010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:43.490010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 352/698] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:43.498174 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebc831-f2ad-4419-ad55-ff517bf0a28a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:43.606798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebc831-f2ad-4419-ad55-ff517bf0a28a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:43.616696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:43.646710 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f8102-93c2-448d-9ee0-a9e5227e8ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:43.647325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f8102-93c2-448d-9ee0-a9e5227e8ccb None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:43.647601 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f8102-93c2-448d-9ee0-a9e5227e8ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:43.647872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f8102-93c2-448d-9ee0-a9e5227e8ccb None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:43.648546 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f8102-93c2-448d-9ee0-a9e5227e8ccb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:43.648957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f8102-93c2-448d-9ee0-a9e5227e8ccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:43.650376 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f8102-93c2-448d-9ee0-a9e5227e8ccb None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:43.660203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 347/699] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:43.665709 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c20f01-6f8f-41af-a110-06bebcce89dc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:43.779112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c20f01-6f8f-41af-a110-06bebcce89dc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:43.821244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8e680-484f-48d2-9d8a-0532f6bccb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:43.821953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8e680-484f-48d2-9d8a-0532f6bccb6c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:43.822311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8e680-484f-48d2-9d8a-0532f6bccb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:43.822593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8e680-484f-48d2-9d8a-0532f6bccb6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:43.836148 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8e680-484f-48d2-9d8a-0532f6bccb6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:43.837614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8e680-484f-48d2-9d8a-0532f6bccb6c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-a5f8e680-484f-48d2-9d8a-0532f6bccb6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:43.853883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:43.856453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:43.857700 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:43.857700 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:43.857700 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:43.857700 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 353/700] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:43.861737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f8102-93c2-448d-9ee0-a9e5227e8ccb None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:43.949575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5353d21-8e9f-48f8-8879-93d9e1cff1b4 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:43.950127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5353d21-8e9f-48f8-8879-93d9e1cff1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:43.950417 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5353d21-8e9f-48f8-8879-93d9e1cff1b4 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:43.952722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 354/701] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:11:43.959754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb03d95-e586-403f-b7e3-76c7b1ea116b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:43.960305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb03d95-e586-403f-b7e3-76c7b1ea116b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:43.960775 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb03d95-e586-403f-b7e3-76c7b1ea116b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:43.987740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2cb03d95-e586-403f-b7e3-76c7b1ea116b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:43.989746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2cb03d95-e586-403f-b7e3-76c7b1ea116b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:44.045855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f8102-93c2-448d-9ee0-a9e5227e8ccb None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:44.065634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:44.114453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 355/702] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:11:44.121176 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdacbf58-ae12-401c-b5d4-b34b2b72c7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:44.121793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacbf58-ae12-401c-b5d4-b34b2b72c7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:44.122085 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacbf58-ae12-401c-b5d4-b34b2b72c7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:44.122359 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdacbf58-ae12-401c-b5d4-b34b2b72c7e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:44.123030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdacbf58-ae12-401c-b5d4-b34b2b72c7e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:44.123314 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdacbf58-ae12-401c-b5d4-b34b2b72c7e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:44.124675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdacbf58-ae12-401c-b5d4-b34b2b72c7e3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:44.145736 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 348/703] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:44.150821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb03d95-e586-403f-b7e3-76c7b1ea116b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:44.301108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb03d95-e586-403f-b7e3-76c7b1ea116b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:44.322384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d1c08-2d20-40fe-83cb-b5b4967ab94f None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:44.322670 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d1c08-2d20-40fe-83cb-b5b4967ab94f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:44.322942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d1c08-2d20-40fe-83cb-b5b4967ab94f None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:44.324502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 349/704] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:44.329695 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5f0f7-d57d-44cc-9c97-9a22b3db7434 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:44.329964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5f0f7-d57d-44cc-9c97-9a22b3db7434 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:44.330280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5f0f7-d57d-44cc-9c97-9a22b3db7434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:44.349085 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f8b5f0f7-d57d-44cc-9c97-9a22b3db7434 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:44.350430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f8b5f0f7-d57d-44cc-9c97-9a22b3db7434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:44.355046 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a44bb3-8ff1-4d4d-8303-b1c1cd441ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:44.355964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a44bb3-8ff1-4d4d-8303-b1c1cd441ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:44.356234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a44bb3-8ff1-4d4d-8303-b1c1cd441ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:44.356519 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a44bb3-8ff1-4d4d-8303-b1c1cd441ac2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:44.370033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a44bb3-8ff1-4d4d-8303-b1c1cd441ac2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:44.371482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a44bb3-8ff1-4d4d-8303-b1c1cd441ac2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c4a44bb3-8ff1-4d4d-8303-b1c1cd441ac2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:44.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:44.396020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:44.396965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:44.396965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:44.396965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:44.396965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:44.396965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:44.396965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:44.396965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:44.396965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:44.396965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:11:44.396965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:44.396965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 356/705] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Sep 27 17:11:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:44.400388 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:44.402732 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a44bb3-8ff1-4d4d-8303-b1c1cd441ac2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:44.447855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:44.448152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 350/706] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 27 17:11:44.508828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5f0f7-d57d-44cc-9c97-9a22b3db7434 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:44.511292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a44bb3-8ff1-4d4d-8303-b1c1cd441ac2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:44.546289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adf98a-3242-4bb2-a5aa-99228d0261fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:44.546889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adf98a-3242-4bb2-a5aa-99228d0261fd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:44.550349 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adf98a-3242-4bb2-a5aa-99228d0261fd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:44.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adf98a-3242-4bb2-a5aa-99228d0261fd None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:44.552262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12adf98a-3242-4bb2-a5aa-99228d0261fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:44.552538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12adf98a-3242-4bb2-a5aa-99228d0261fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:44.555436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12adf98a-3242-4bb2-a5aa-99228d0261fd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:44.594253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 357/707] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Sep 27 17:11:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:44.600879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adf98a-3242-4bb2-a5aa-99228d0261fd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:44.713138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5f0f7-d57d-44cc-9c97-9a22b3db7434 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:44.738617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:44.742817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adf98a-3242-4bb2-a5aa-99228d0261fd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:44.794777 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f50337-e847-4b04-977b-e24d33bcff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:44.794994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffd8e2-c5f7-44c0-ab58-1a35ef279540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:44.795426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f50337-e847-4b04-977b-e24d33bcff09 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:44.795536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffd8e2-c5f7-44c0-ab58-1a35ef279540 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:44.795692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f50337-e847-4b04-977b-e24d33bcff09 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:44.795785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffd8e2-c5f7-44c0-ab58-1a35ef279540 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:44.795964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f50337-e847-4b04-977b-e24d33bcff09 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:44.796075 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ffd8e2-c5f7-44c0-ab58-1a35ef279540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:44.796553 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f50337-e847-4b04-977b-e24d33bcff09 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:44.797983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f50337-e847-4b04-977b-e24d33bcff09 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-e6f50337-e847-4b04-977b-e24d33bcff09 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:44.811981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:44.813034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 17:11:44.814474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:44.818185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 351/708] 10.222.0.55 () {56 vars in 1129 bytes} [Tue Sep 27 17:11:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:44.819040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f50337-e847-4b04-977b-e24d33bcff09 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:44.850001 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:44.883189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffd8e2-c5f7-44c0-ab58-1a35ef279540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:44.884935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ffd8e2-c5f7-44c0-ab58-1a35ef279540 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:44.927995 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f50337-e847-4b04-977b-e24d33bcff09 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:44.967834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ed56c-1c22-4f6e-8def-63261abb0414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:44.968464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ed56c-1c22-4f6e-8def-63261abb0414 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:44.968739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ed56c-1c22-4f6e-8def-63261abb0414 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:44.969005 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ed56c-1c22-4f6e-8def-63261abb0414 None admin] PATH_INFO: `/v3/services/network` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:44.970468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ed56c-1c22-4f6e-8def-63261abb0414 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:44.972750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ed56c-1c22-4f6e-8def-63261abb0414 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-fe5ed56c-1c22-4f6e-8def-63261abb0414 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:44.985712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:44.986919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 17:11:44.987900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:44.989082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:44.991635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 352/709] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:44 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:44.992942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ed56c-1c22-4f6e-8def-63261abb0414 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:44.999575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 358/710] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:45.000602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffd8e2-c5f7-44c0-ab58-1a35ef279540 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:45.113941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:45.122485 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ffd8e2-c5f7-44c0-ab58-1a35ef279540 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:45.140183 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ed56c-1c22-4f6e-8def-63261abb0414 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:45.163052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eac3d7-06de-493e-92d4-fe40130ce321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:45.164159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eac3d7-06de-493e-92d4-fe40130ce321 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:45.164668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eac3d7-06de-493e-92d4-fe40130ce321 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:45.165141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eac3d7-06de-493e-92d4-fe40130ce321 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:45.165975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eac3d7-06de-493e-92d4-fe40130ce321 None admin] RBAC: Adding query filter params (name=network) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:45.166504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eac3d7-06de-493e-92d4-fe40130ce321 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:45.168078 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eac3d7-06de-493e-92d4-fe40130ce321 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:45.174780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 359/711] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Sep 27 17:11:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:45.175854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eac3d7-06de-493e-92d4-fe40130ce321 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:45.187371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73a9df-4212-486c-898b-9cb391baa208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:45.190700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73a9df-4212-486c-898b-9cb391baa208 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:45.190991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73a9df-4212-486c-898b-9cb391baa208 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:45.191267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73a9df-4212-486c-898b-9cb391baa208 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:45.191916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73a9df-4212-486c-898b-9cb391baa208 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:45.192220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73a9df-4212-486c-898b-9cb391baa208 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:45.194257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73a9df-4212-486c-898b-9cb391baa208 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:45.200632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 353/712] 10.222.0.55 () {56 vars in 1124 bytes} [Tue Sep 27 17:11:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:45.202050 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73a9df-4212-486c-898b-9cb391baa208 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:45.247143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:45.293004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eac3d7-06de-493e-92d4-fe40130ce321 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:45.309161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73a9df-4212-486c-898b-9cb391baa208 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:45.335795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be3b6d-55f6-41ed-b139-851319d578d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:45.336424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be3b6d-55f6-41ed-b139-851319d578d5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:45.336698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be3b6d-55f6-41ed-b139-851319d578d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:45.336969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be3b6d-55f6-41ed-b139-851319d578d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:45.345217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097e3ab-e2b4-49a3-959f-cff1205e5416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:45.346018 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097e3ab-e2b4-49a3-959f-cff1205e5416 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:45.346480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097e3ab-e2b4-49a3-959f-cff1205e5416 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:45.346895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be3b6d-55f6-41ed-b139-851319d578d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:45.347030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097e3ab-e2b4-49a3-959f-cff1205e5416 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:45.347778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097e3ab-e2b4-49a3-959f-cff1205e5416 None admin] RBAC: Adding query filter params (type=network) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:45.348233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097e3ab-e2b4-49a3-959f-cff1205e5416 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:45.348357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be3b6d-55f6-41ed-b139-851319d578d5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:45.350141 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097e3ab-e2b4-49a3-959f-cff1205e5416 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:45.356516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 354/713] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Sep 27 17:11:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:45.357935 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e86d29-08f7-4e55-a88f-684f16955b89 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:45.358247 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e86d29-08f7-4e55-a88f-684f16955b89 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:45.358524 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e86d29-08f7-4e55-a88f-684f16955b89 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:45.359917 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 355/714] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:45.365444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e86d29-08f7-4e55-a88f-684f16955b89 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d5be3b6d-55f6-41ed-b139-851319d578d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:45.373491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:45.374586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:45.375555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:45.375555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:45.375555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:45.378178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 360/715] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:45.379707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5ed8e-afe5-4e89-a1a8-b63b88e506c4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:45.380230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5ed8e-afe5-4e89-a1a8-b63b88e506c4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:45.380697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5ed8e-afe5-4e89-a1a8-b63b88e506c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:45.400104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0df5ed8e-afe5-4e89-a1a8-b63b88e506c4 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:45.401668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0df5ed8e-afe5-4e89-a1a8-b63b88e506c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:45.402576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:45.496124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 361/716] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:45.497523 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5ed8e-afe5-4e89-a1a8-b63b88e506c4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:45.520953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e86d29-08f7-4e55-a88f-684f16955b89 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:45.572161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:45.580662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d74545-f931-4831-ae78-21bd81e134a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:45.581559 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d74545-f931-4831-ae78-21bd81e134a5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:45.582032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d74545-f931-4831-ae78-21bd81e134a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:45.582530 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d74545-f931-4831-ae78-21bd81e134a5 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:45.583531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d74545-f931-4831-ae78-21bd81e134a5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:45.583838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d74545-f931-4831-ae78-21bd81e134a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:45.585205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d74545-f931-4831-ae78-21bd81e134a5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:45.591984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 356/717] 10.222.0.55 () {56 vars in 1124 bytes} [Tue Sep 27 17:11:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:45.642857 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5ed8e-afe5-4e89-a1a8-b63b88e506c4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:45.683253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4d433-a22b-43f2-a4d6-9cdf51d18ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:45.684242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d433-a22b-43f2-a4d6-9cdf51d18ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:45.684770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d433-a22b-43f2-a4d6-9cdf51d18ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:45.685268 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d433-a22b-43f2-a4d6-9cdf51d18ed8 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:45.686229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4d433-a22b-43f2-a4d6-9cdf51d18ed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:45.686752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4d433-a22b-43f2-a4d6-9cdf51d18ed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:45.688414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4d433-a22b-43f2-a4d6-9cdf51d18ed8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:45.702472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 362/718] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:45.710115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d74545-f931-4831-ae78-21bd81e134a5 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:45.819834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:45.821558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4d433-a22b-43f2-a4d6-9cdf51d18ed8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:45.843484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d74545-f931-4831-ae78-21bd81e134a5 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:45.889502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85402df7-3e2b-4922-8db6-8934481457e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:45.890164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85402df7-3e2b-4922-8db6-8934481457e6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:45.890500 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85402df7-3e2b-4922-8db6-8934481457e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:45.890784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85402df7-3e2b-4922-8db6-8934481457e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:45.898485 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85402df7-3e2b-4922-8db6-8934481457e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:45.900219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85402df7-3e2b-4922-8db6-8934481457e6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-85402df7-3e2b-4922-8db6-8934481457e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:45.912630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:45.913855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:45.915079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:45.915079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:45.915079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:45.915079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:45.915079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:45.915079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:45.915079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:45.915079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:45.918170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 357/719] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:45.922337 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85402df7-3e2b-4922-8db6-8934481457e6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:46.020561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4d433-a22b-43f2-a4d6-9cdf51d18ed8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:46.023088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:46.036359 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85402df7-3e2b-4922-8db6-8934481457e6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:46.080606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b383ce0-d241-463a-a0dd-f77e93d8af07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:46.081271 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b383ce0-d241-463a-a0dd-f77e93d8af07 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:46.081557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b383ce0-d241-463a-a0dd-f77e93d8af07 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:46.082549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b383ce0-d241-463a-a0dd-f77e93d8af07 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:46.083258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b383ce0-d241-463a-a0dd-f77e93d8af07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:46.085560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b383ce0-d241-463a-a0dd-f77e93d8af07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:46.085560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b383ce0-d241-463a-a0dd-f77e93d8af07 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:46.102588 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2b96a-0a7e-4a4f-b674-765db0234022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:46.103220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2b96a-0a7e-4a4f-b674-765db0234022 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:46.103500 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2b96a-0a7e-4a4f-b674-765db0234022 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:46.103772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2b96a-0a7e-4a4f-b674-765db0234022 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:46.109377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:46.115240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 358/720] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:46.121090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b383ce0-d241-463a-a0dd-f77e93d8af07 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:46.138785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2b96a-0a7e-4a4f-b674-765db0234022 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-cfb2b96a-0a7e-4a4f-b674-765db0234022 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:46.144668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:46.145624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:46.147190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:46.147190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:46.147190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:46.147190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:46.147190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:46.147190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:46.147190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:46.147190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:46.147190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:46.147190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:46.147190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:46.147190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:46.147190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 363/721] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:46.152624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2b96a-0a7e-4a4f-b674-765db0234022 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:46.247309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b383ce0-d241-463a-a0dd-f77e93d8af07 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:46.260889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2b96a-0a7e-4a4f-b674-765db0234022 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:46.290482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8156a-52ae-42d7-b244-d8eef8b4d56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:46.291481 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8156a-52ae-42d7-b244-d8eef8b4d56e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:46.291929 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8156a-52ae-42d7-b244-d8eef8b4d56e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:46.292379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8156a-52ae-42d7-b244-d8eef8b4d56e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:46.293618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8156a-52ae-42d7-b244-d8eef8b4d56e None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=5537185f42474cfd86fca70fa8beddd2) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:46.293932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8156a-52ae-42d7-b244-d8eef8b4d56e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:46.295404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8156a-52ae-42d7-b244-d8eef8b4d56e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:46.301031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 359/722] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:46 2022] GET /identity/v3/role_assignments?user.id=5537185f42474cfd86fca70fa8beddd2&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:46.302250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2017f6-2808-4007-b86b-4c684fde14d8 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:46.302527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2017f6-2808-4007-b86b-4c684fde14d8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:46.302797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2017f6-2808-4007-b86b-4c684fde14d8 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:46.308485 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 360/723] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:46.309683 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f68da3-d998-4764-9727-829ef64931c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:46.309888 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3d037-5a71-4025-b037-07742d43661b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:46.310142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3d037-5a71-4025-b037-07742d43661b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:46.310330 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f68da3-d998-4764-9727-829ef64931c2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:46.310423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3d037-5a71-4025-b037-07742d43661b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:46.310611 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f68da3-d998-4764-9727-829ef64931c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:46.310884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f68da3-d998-4764-9727-829ef64931c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:46.311566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f68da3-d998-4764-9727-829ef64931c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:46.311867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f68da3-d998-4764-9727-829ef64931c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:46.313951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f68da3-d998-4764-9727-829ef64931c2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:46.336331 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-11e3d037-5a71-4025-b037-07742d43661b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:46.337935 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-11e3d037-5a71-4025-b037-07742d43661b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:46.348546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 364/724] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:46.356369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f68da3-d998-4764-9727-829ef64931c2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:46.398470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:46.462540 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:46.479145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 361/725] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:46.537701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f68da3-d998-4764-9727-829ef64931c2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:46.580522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b09bd-1d41-4638-a9c8-8920249350fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:46.581170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b09bd-1d41-4638-a9c8-8920249350fb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:46.581501 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b09bd-1d41-4638-a9c8-8920249350fb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:46.581781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b09bd-1d41-4638-a9c8-8920249350fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:46.594184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b09bd-1d41-4638-a9c8-8920249350fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:46.596166 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b09bd-1d41-4638-a9c8-8920249350fb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-001b09bd-1d41-4638-a9c8-8920249350fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:46.610908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:46.611886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:46.612808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:46.612808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:46.612808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:46.612808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 365/726] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:46.619026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3d037-5a71-4025-b037-07742d43661b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:46.733246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3d037-5a71-4025-b037-07742d43661b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:46.782963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3fe33-7095-4e45-86be-2b25615a0ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:46.783570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3fe33-7095-4e45-86be-2b25615a0ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:46.783851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3fe33-7095-4e45-86be-2b25615a0ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:46.784123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3fe33-7095-4e45-86be-2b25615a0ce7 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:46.784769 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3fe33-7095-4e45-86be-2b25615a0ce7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:46.785071 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3fe33-7095-4e45-86be-2b25615a0ce7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:46.786581 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3fe33-7095-4e45-86be-2b25615a0ce7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:46.799257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 362/727] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:46.807199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b09bd-1d41-4638-a9c8-8920249350fb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:46.846059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3fe33-7095-4e45-86be-2b25615a0ce7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:46.906862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:46.934445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:46.977978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b09bd-1d41-4638-a9c8-8920249350fb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:46.995289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3fe33-7095-4e45-86be-2b25615a0ce7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:47.031299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad41a14-fb42-4d62-a744-b93264c0fa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:47.032597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad41a14-fb42-4d62-a744-b93264c0fa45 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:47.032877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad41a14-fb42-4d62-a744-b93264c0fa45 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:47.033252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad41a14-fb42-4d62-a744-b93264c0fa45 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:47.041499 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad41a14-fb42-4d62-a744-b93264c0fa45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:47.043181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad41a14-fb42-4d62-a744-b93264c0fa45 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-9ad41a14-fb42-4d62-a744-b93264c0fa45 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:47.054964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:47.055935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:47.057631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:47.057631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:47.057631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:47.057631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:47.057631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:47.057631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:47.057631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:47.057631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:47.057631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 17:11:47.057631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:47.057631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 366/728] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Sep 27 17:11:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:47.061775 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad41a14-fb42-4d62-a744-b93264c0fa45 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:47.065558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae764e9c-0826-4bd6-8064-298ab2d69e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:47.066234 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae764e9c-0826-4bd6-8064-298ab2d69e53 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:47.066517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae764e9c-0826-4bd6-8064-298ab2d69e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:47.066789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae764e9c-0826-4bd6-8064-298ab2d69e53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:47.087615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae764e9c-0826-4bd6-8064-298ab2d69e53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ae764e9c-0826-4bd6-8064-298ab2d69e53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:47.093715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:47.096490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:47.098489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:47.098489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:47.098489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:47.098489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:47.098489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:47.098489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:47.098489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:47.098489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:47.098489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:47.098489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:47.098489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:47.098489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:47.098489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 363/729] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:47.106675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae764e9c-0826-4bd6-8064-298ab2d69e53 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:47.169597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad41a14-fb42-4d62-a744-b93264c0fa45 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:47.205983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a3a42-66ca-4cb2-b26d-a3c5f416e863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:47.206624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a3a42-66ca-4cb2-b26d-a3c5f416e863 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:47.206903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a3a42-66ca-4cb2-b26d-a3c5f416e863 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:47.207173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a3a42-66ca-4cb2-b26d-a3c5f416e863 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:47.209294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a3a42-66ca-4cb2-b26d-a3c5f416e863 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:47.209294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a3a42-66ca-4cb2-b26d-a3c5f416e863 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:47.209621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a3a42-66ca-4cb2-b26d-a3c5f416e863 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:47.212360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:47.230731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 367/730] 10.222.0.55 () {56 vars in 1149 bytes} [Tue Sep 27 17:11:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:47.236561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a3a42-66ca-4cb2-b26d-a3c5f416e863 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:47.331237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae764e9c-0826-4bd6-8064-298ab2d69e53 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:47.354765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a3a42-66ca-4cb2-b26d-a3c5f416e863 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:47.361126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:47.394151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f96a1-0e86-4861-8773-e40814d25dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:47.395128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f96a1-0e86-4861-8773-e40814d25dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:47.395589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f96a1-0e86-4861-8773-e40814d25dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:47.396115 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f96a1-0e86-4861-8773-e40814d25dc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:47.404022 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f01e36-7d6b-4cc6-9ca3-aeb8cdfce3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:47.409651 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f01e36-7d6b-4cc6-9ca3-aeb8cdfce3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:47.410185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f01e36-7d6b-4cc6-9ca3-aeb8cdfce3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:47.410678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f01e36-7d6b-4cc6-9ca3-aeb8cdfce3f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:47.411562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f01e36-7d6b-4cc6-9ca3-aeb8cdfce3f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:47.412064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f01e36-7d6b-4cc6-9ca3-aeb8cdfce3f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:47.413638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f01e36-7d6b-4cc6-9ca3-aeb8cdfce3f4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:47.431438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 364/731] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:47.437785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f01e36-7d6b-4cc6-9ca3-aeb8cdfce3f4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:47.498334 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:47.509704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f96a1-0e86-4861-8773-e40814d25dc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:47.511611 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f96a1-0e86-4861-8773-e40814d25dc2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:47.616043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 368/732] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 27 17:11:47.621915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f96a1-0e86-4861-8773-e40814d25dc2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:47.644309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f01e36-7d6b-4cc6-9ca3-aeb8cdfce3f4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:47.719726 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:47.726213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f96a1-0e86-4861-8773-e40814d25dc2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:47.758770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea0d87-d52d-4fc2-a516-0397a0f891e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:47.759410 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea0d87-d52d-4fc2-a516-0397a0f891e2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:47.759903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea0d87-d52d-4fc2-a516-0397a0f891e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:47.760229 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea0d87-d52d-4fc2-a516-0397a0f891e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:47.764300 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3aa70e-dd5f-4ebb-9c8f-b055e6971329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:47.765169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3aa70e-dd5f-4ebb-9c8f-b055e6971329 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:47.765647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3aa70e-dd5f-4ebb-9c8f-b055e6971329 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:47.766678 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3aa70e-dd5f-4ebb-9c8f-b055e6971329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:47.777613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3aa70e-dd5f-4ebb-9c8f-b055e6971329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:47.779560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3aa70e-dd5f-4ebb-9c8f-b055e6971329 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:47.791048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea0d87-d52d-4fc2-a516-0397a0f891e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:47.792506 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea0d87-d52d-4fc2-a516-0397a0f891e2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-9b3aa70e-dd5f-4ebb-9c8f-b055e6971329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:47.795385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:47.796313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:47.797283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:47.797283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:47.797283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:47.799390 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 369/733] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:47.805093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3aa70e-dd5f-4ebb-9c8f-b055e6971329 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-06ea0d87-d52d-4fc2-a516-0397a0f891e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:47.827433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:47.828451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:47.829438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:47.829438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:47.829438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:47.838543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 365/734] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:47.839460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea0d87-d52d-4fc2-a516-0397a0f891e2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:47.900817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:47.936045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3aa70e-dd5f-4ebb-9c8f-b055e6971329 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:47.985915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea0d87-d52d-4fc2-a516-0397a0f891e2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:48.026565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7d9e8-9800-46b2-a2b2-3087e8d42942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:48.027210 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7d9e8-9800-46b2-a2b2-3087e8d42942 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:48.027495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7d9e8-9800-46b2-a2b2-3087e8d42942 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:48.027761 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7d9e8-9800-46b2-a2b2-3087e8d42942 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:48.028446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7d9e8-9800-46b2-a2b2-3087e8d42942 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:48.028753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7d9e8-9800-46b2-a2b2-3087e8d42942 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:48.030208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7d9e8-9800-46b2-a2b2-3087e8d42942 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:48.034558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06278e3-cb15-45e8-9451-1456ee7928d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:48.035223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06278e3-cb15-45e8-9451-1456ee7928d2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:48.035458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06278e3-cb15-45e8-9451-1456ee7928d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:48.035729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06278e3-cb15-45e8-9451-1456ee7928d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:48.036392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06278e3-cb15-45e8-9451-1456ee7928d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:48.036690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06278e3-cb15-45e8-9451-1456ee7928d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:48.038149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06278e3-cb15-45e8-9451-1456ee7928d2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:48.058387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 366/735] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:48.067105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7d9e8-9800-46b2-a2b2-3087e8d42942 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:48.073295 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 370/736] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:48.079784 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06278e3-cb15-45e8-9451-1456ee7928d2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:48.091072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:48.211415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:48.223708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06278e3-cb15-45e8-9451-1456ee7928d2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:48.224133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7d9e8-9800-46b2-a2b2-3087e8d42942 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:48.260256 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06c1a8-e72d-45e1-b159-cf5d6a0ac81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:48.260869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06c1a8-e72d-45e1-b159-cf5d6a0ac81b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:48.261148 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06c1a8-e72d-45e1-b159-cf5d6a0ac81b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:48.261417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06c1a8-e72d-45e1-b159-cf5d6a0ac81b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:48.262257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ede43-7074-441e-a20f-6bc824329c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:48.262873 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ede43-7074-441e-a20f-6bc824329c54 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:48.263153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ede43-7074-441e-a20f-6bc824329c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:48.263423 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ede43-7074-441e-a20f-6bc824329c54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:48.268839 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06c1a8-e72d-45e1-b159-cf5d6a0ac81b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:48.270063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ede43-7074-441e-a20f-6bc824329c54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:48.270294 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06c1a8-e72d-45e1-b159-cf5d6a0ac81b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:48.271488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ede43-7074-441e-a20f-6bc824329c54 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-1c06c1a8-e72d-45e1-b159-cf5d6a0ac81b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:48.281548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:48.282521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:48.283464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:48.283464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:48.283464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:48.283464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:48.283464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:48.283464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:48.283464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:48.283464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:48.283464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 27 17:11:48.283464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-fe9ede43-7074-441e-a20f-6bc824329c54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:48.284027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:48.284845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:48.285757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:48.285757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:48.285757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:48.285757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:48.285757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:48.285757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:48.285757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:48.285757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:48.287599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 367/737] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:48.288666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6047c-5030-43bb-a249-4208fff0b6f3 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:48.289118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6047c-5030-43bb-a249-4208fff0b6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:48.289281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 371/738] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:48.289441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a6047c-5030-43bb-a249-4208fff0b6f3 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:48.290503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b24fa-99ce-4f69-b56d-5c2c1773412b None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:48.290868 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b24fa-99ce-4f69-b56d-5c2c1773412b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:48.291029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 368/739] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:48.291171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b24fa-99ce-4f69-b56d-5c2c1773412b None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:48.295459 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6047c-5030-43bb-a249-4208fff0b6f3 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:48.295658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 372/740] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:48.295658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b24fa-99ce-4f69-b56d-5c2c1773412b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:48.373932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:48.402536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a6047c-5030-43bb-a249-4208fff0b6f3 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:48.413418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b24fa-99ce-4f69-b56d-5c2c1773412b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:48.436288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705bd03-7181-4247-9918-844f072c3d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:48.436886 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705bd03-7181-4247-9918-844f072c3d39 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:48.437165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705bd03-7181-4247-9918-844f072c3d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:48.437435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705bd03-7181-4247-9918-844f072c3d39 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:48.438298 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705bd03-7181-4247-9918-844f072c3d39 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:48.438836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705bd03-7181-4247-9918-844f072c3d39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:48.440023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705bd03-7181-4247-9918-844f072c3d39 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:48.451244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1688ac-5e81-451e-ab79-8e8970cd3831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:48.451845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1688ac-5e81-451e-ab79-8e8970cd3831 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:48.452120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1688ac-5e81-451e-ab79-8e8970cd3831 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:48.452388 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1688ac-5e81-451e-ab79-8e8970cd3831 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:48.453077 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1688ac-5e81-451e-ab79-8e8970cd3831 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:48.453381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1688ac-5e81-451e-ab79-8e8970cd3831 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:48.455950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1688ac-5e81-451e-ab79-8e8970cd3831 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:48.463246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 369/741] 10.222.0.55 () {56 vars in 1135 bytes} [Tue Sep 27 17:11:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:48.463246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf022d8-ee2f-4df7-ac29-836346747d54 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:48.463246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf022d8-ee2f-4df7-ac29-836346747d54 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:48.463246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf022d8-ee2f-4df7-ac29-836346747d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:48.468744 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 373/742] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:48.473290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cae97-7662-4bc3-bcdf-7aee083f68e9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:48.473290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cae97-7662-4bc3-bcdf-7aee083f68e9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:48.473290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cae97-7662-4bc3-bcdf-7aee083f68e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:48.475343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:48.479842 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ebf022d8-ee2f-4df7-ac29-836346747d54 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:48.482729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ebf022d8-ee2f-4df7-ac29-836346747d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:48.491542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c97cae97-7662-4bc3-bcdf-7aee083f68e9 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:48.492718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c97cae97-7662-4bc3-bcdf-7aee083f68e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:48.553655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 370/743] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:48.555644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf022d8-ee2f-4df7-ac29-836346747d54 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:48.574096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 374/744] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 27 17:11:48.574096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cae97-7662-4bc3-bcdf-7aee083f68e9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:48.646371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:48.693318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cae97-7662-4bc3-bcdf-7aee083f68e9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:48.732178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f03d19-d542-4d51-b1d8-bfcb417caaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:48.733448 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f03d19-d542-4d51-b1d8-bfcb417caaef None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:48.733727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f03d19-d542-4d51-b1d8-bfcb417caaef None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:48.733997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f03d19-d542-4d51-b1d8-bfcb417caaef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:48.734747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f03d19-d542-4d51-b1d8-bfcb417caaef None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=c8c3fcd3789c40c88a4c32fff2678fc9) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:48.739910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f03d19-d542-4d51-b1d8-bfcb417caaef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:48.742968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f03d19-d542-4d51-b1d8-bfcb417caaef None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:48.752747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 375/745] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:48 2022] GET /identity/v3/role_assignments?user.id=c8c3fcd3789c40c88a4c32fff2678fc9&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:48.777374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf022d8-ee2f-4df7-ac29-836346747d54 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:48.819228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48704fe-adad-4d16-8497-49bd15319f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:48.819846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48704fe-adad-4d16-8497-49bd15319f18 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:48.820115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48704fe-adad-4d16-8497-49bd15319f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:48.820384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48704fe-adad-4d16-8497-49bd15319f18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:48.824155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:48.874355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f03d19-d542-4d51-b1d8-bfcb417caaef None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:48.913366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48704fe-adad-4d16-8497-49bd15319f18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:48.915044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48704fe-adad-4d16-8497-49bd15319f18 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:49.020517 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f03d19-d542-4d51-b1d8-bfcb417caaef None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:49.041049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 371/746] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 27 17:11:49.042235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:49.046424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48704fe-adad-4d16-8497-49bd15319f18 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:49.058309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11d9a9-3449-4ad4-998b-e044b6bd6825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:49.058907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11d9a9-3449-4ad4-998b-e044b6bd6825 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:49.059182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11d9a9-3449-4ad4-998b-e044b6bd6825 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:49.059449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11d9a9-3449-4ad4-998b-e044b6bd6825 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:49.060095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11d9a9-3449-4ad4-998b-e044b6bd6825 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:49.061435 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11d9a9-3449-4ad4-998b-e044b6bd6825 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:49.086471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 376/747] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:11:48 2022] POST /identity/v3/services => generated 240 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:11:49.087871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af41c5-12f2-4803-9633-873baee5f830 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:49.088157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af41c5-12f2-4803-9633-873baee5f830 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:49.088427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af41c5-12f2-4803-9633-873baee5f830 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:49.090009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 377/748] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:49.091676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:49.096004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af41c5-12f2-4803-9633-873baee5f830 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:49.201372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48704fe-adad-4d16-8497-49bd15319f18 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:49.253127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af41c5-12f2-4803-9633-873baee5f830 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:49.259534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98002eb-8c56-485a-be24-2e5bd7ce6bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:49.260142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98002eb-8c56-485a-be24-2e5bd7ce6bb7 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:49.260414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98002eb-8c56-485a-be24-2e5bd7ce6bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:49.260677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98002eb-8c56-485a-be24-2e5bd7ce6bb7 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:49.261320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98002eb-8c56-485a-be24-2e5bd7ce6bb7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:49.263183 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98002eb-8c56-485a-be24-2e5bd7ce6bb7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:49.288199 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 372/749] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:11:49 2022] POST /identity/v3/services => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:11:49.289362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b395b7-97fd-4f79-8b52-3c3de58c4a3a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:49.289648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b395b7-97fd-4f79-8b52-3c3de58c4a3a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:49.289918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b395b7-97fd-4f79-8b52-3c3de58c4a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:49.320211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c1b395b7-97fd-4f79-8b52-3c3de58c4a3a None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:49.321055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812bdf6-fe4b-4286-bea9-41a4d162c176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:49.321490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c1b395b7-97fd-4f79-8b52-3c3de58c4a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:49.322193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812bdf6-fe4b-4286-bea9-41a4d162c176 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:49.322676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812bdf6-fe4b-4286-bea9-41a4d162c176 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:49.323149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812bdf6-fe4b-4286-bea9-41a4d162c176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:49.338568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812bdf6-fe4b-4286-bea9-41a4d162c176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:49.340647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812bdf6-fe4b-4286-bea9-41a4d162c176 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c812bdf6-fe4b-4286-bea9-41a4d162c176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:49.361763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:49.362812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:49.363733 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:49.363733 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:49.363733 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:49.366932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 378/750] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:49.375055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812bdf6-fe4b-4286-bea9-41a4d162c176 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:49.401630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 373/751] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:11:49.404792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:49.486992 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812bdf6-fe4b-4286-bea9-41a4d162c176 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:49.523659 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de97b98-1736-44c2-bdfc-935a791a6826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:49.524259 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de97b98-1736-44c2-bdfc-935a791a6826 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:49.524532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de97b98-1736-44c2-bdfc-935a791a6826 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:49.524790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de97b98-1736-44c2-bdfc-935a791a6826 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:49.525489 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de97b98-1736-44c2-bdfc-935a791a6826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:49.525785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de97b98-1736-44c2-bdfc-935a791a6826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:49.527178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de97b98-1736-44c2-bdfc-935a791a6826 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:49.543698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 379/752] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:49.550264 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b395b7-97fd-4f79-8b52-3c3de58c4a3a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:49.580365 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:49.677956 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b395b7-97fd-4f79-8b52-3c3de58c4a3a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:49.715463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237d6aa-329e-429c-a6dc-4ae0ce88cba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:49.716092 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237d6aa-329e-429c-a6dc-4ae0ce88cba8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:49.716362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237d6aa-329e-429c-a6dc-4ae0ce88cba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:49.716625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237d6aa-329e-429c-a6dc-4ae0ce88cba8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:49.724019 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237d6aa-329e-429c-a6dc-4ae0ce88cba8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:49.725484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237d6aa-329e-429c-a6dc-4ae0ce88cba8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-c237d6aa-329e-429c-a6dc-4ae0ce88cba8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:49.737242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:49.738187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:49.739082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:49.739082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:49.739082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:49.739082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:49.739082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:49.739082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:49.739082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:49.739082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:49.739082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 374/753] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:49.744633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de97b98-1736-44c2-bdfc-935a791a6826 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:49.748224 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237d6aa-329e-429c-a6dc-4ae0ce88cba8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:49.854514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:49.856100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:49.867296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237d6aa-329e-429c-a6dc-4ae0ce88cba8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:49.877359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de97b98-1736-44c2-bdfc-935a791a6826 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:49.903159 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54128a37-5a2b-4bfe-b8a4-2de921278e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:49.903787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54128a37-5a2b-4bfe-b8a4-2de921278e6b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:49.904059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54128a37-5a2b-4bfe-b8a4-2de921278e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:49.904323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54128a37-5a2b-4bfe-b8a4-2de921278e6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:49.914918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54128a37-5a2b-4bfe-b8a4-2de921278e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:49.919044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1d1ad-7745-4cb5-8c80-77e4f0573cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:49.919655 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1d1ad-7745-4cb5-8c80-77e4f0573cdc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:49.919928 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1d1ad-7745-4cb5-8c80-77e4f0573cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:49.920196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1d1ad-7745-4cb5-8c80-77e4f0573cdc None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:49.920885 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1d1ad-7745-4cb5-8c80-77e4f0573cdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-54128a37-5a2b-4bfe-b8a4-2de921278e6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:49.921098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:49.922763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:49.923716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:49.923716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:49.923716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:49.923716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:49.923716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:49.923716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:49.923716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:49.923716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:49.923716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:49.923716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:49.923716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:49.923716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:49.924436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1d1ad-7745-4cb5-8c80-77e4f0573cdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:49.924522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 375/754] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:49.924593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1d1ad-7745-4cb5-8c80-77e4f0573cdc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:49.938663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54128a37-5a2b-4bfe-b8a4-2de921278e6b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:49.939871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 380/755] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:49.946684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1d1ad-7745-4cb5-8c80-77e4f0573cdc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:50.097103 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54128a37-5a2b-4bfe-b8a4-2de921278e6b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:50.121710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1d1ad-7745-4cb5-8c80-77e4f0573cdc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:50.131602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19146ec7-e57d-457e-ab3a-71a72aedf773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:50.132258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19146ec7-e57d-457e-ab3a-71a72aedf773 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:50.132460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19146ec7-e57d-457e-ab3a-71a72aedf773 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:50.132724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19146ec7-e57d-457e-ab3a-71a72aedf773 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:50.133378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19146ec7-e57d-457e-ab3a-71a72aedf773 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:50.133671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19146ec7-e57d-457e-ab3a-71a72aedf773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:50.135066 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19146ec7-e57d-457e-ab3a-71a72aedf773 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:50.144367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 376/756] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:50.149868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19146ec7-e57d-457e-ab3a-71a72aedf773 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:50.155238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:50.189491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7ecd6-568f-4f31-954d-759d61c1c1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:50.190128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ecd6-568f-4f31-954d-759d61c1c1cf None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:50.190403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ecd6-568f-4f31-954d-759d61c1c1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:50.190661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7ecd6-568f-4f31-954d-759d61c1c1cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:50.191392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7ecd6-568f-4f31-954d-759d61c1c1cf None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=39073006a0054b56bb1bba46d2d0768e) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:50.191695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7ecd6-568f-4f31-954d-759d61c1c1cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:50.193124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7ecd6-568f-4f31-954d-759d61c1c1cf None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:50.205907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 381/757] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:49 2022] GET /identity/v3/role_assignments?user.id=39073006a0054b56bb1bba46d2d0768e&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:50.305244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19146ec7-e57d-457e-ab3a-71a72aedf773 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:50.359569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cb1d2-78c6-4836-86c1-3e2a9457c6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:50.361188 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cb1d2-78c6-4836-86c1-3e2a9457c6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:50.361674 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cb1d2-78c6-4836-86c1-3e2a9457c6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:50.362212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302cb1d2-78c6-4836-86c1-3e2a9457c6d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:50.374905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cb1d2-78c6-4836-86c1-3e2a9457c6d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:50.376721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302cb1d2-78c6-4836-86c1-3e2a9457c6d1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-302cb1d2-78c6-4836-86c1-3e2a9457c6d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:50.392770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:50.393823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:50.394780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:50.394780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:50.394780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:50.397059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 377/758] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:50.402324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7ecd6-568f-4f31-954d-759d61c1c1cf None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:50.414843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:50.532656 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7ecd6-568f-4f31-954d-759d61c1c1cf None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:50.579204 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a78f9-6874-4bf4-84c9-4b1869a8ebdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:50.580188 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a78f9-6874-4bf4-84c9-4b1869a8ebdf None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:50.580686 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a78f9-6874-4bf4-84c9-4b1869a8ebdf None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:50.581158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a78f9-6874-4bf4-84c9-4b1869a8ebdf None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:50.582003 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a78f9-6874-4bf4-84c9-4b1869a8ebdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:50.582553 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a78f9-6874-4bf4-84c9-4b1869a8ebdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:50.584103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a78f9-6874-4bf4-84c9-4b1869a8ebdf None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:50.597572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 382/759] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:50.602561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cb1d2-78c6-4836-86c1-3e2a9457c6d1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:50.677753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:50.726887 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302cb1d2-78c6-4836-86c1-3e2a9457c6d1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:50.769421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f80baa-3894-4252-9c88-706b57a783ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:50.770050 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f80baa-3894-4252-9c88-706b57a783ce None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:50.770352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f80baa-3894-4252-9c88-706b57a783ce None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:50.770621 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f80baa-3894-4252-9c88-706b57a783ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:50.778339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f80baa-3894-4252-9c88-706b57a783ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:50.779771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f80baa-3894-4252-9c88-706b57a783ce None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-27f80baa-3894-4252-9c88-706b57a783ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:50.791882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:50.792844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:50.794418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:50.794418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:50.794418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:50.794418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:50.794418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:50.794418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:50.794418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:50.794418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:50.794418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:11:50.794418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:50.794418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 378/760] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Sep 27 17:11:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:50.798811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a78f9-6874-4bf4-84c9-4b1869a8ebdf None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:50.910226 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:50.952370 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a78f9-6874-4bf4-84c9-4b1869a8ebdf None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:51.000137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c157f17-c9a4-4a28-9348-d921c3c61491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:51.001323 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c157f17-c9a4-4a28-9348-d921c3c61491 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:51.001604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c157f17-c9a4-4a28-9348-d921c3c61491 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:51.001866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c157f17-c9a4-4a28-9348-d921c3c61491 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:51.002624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c157f17-c9a4-4a28-9348-d921c3c61491 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:51.002919 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c157f17-c9a4-4a28-9348-d921c3c61491 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:51.004343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c157f17-c9a4-4a28-9348-d921c3c61491 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:51.022565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 383/761] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Sep 27 17:11:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:51.028214 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f80baa-3894-4252-9c88-706b57a783ce None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:51.158178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f80baa-3894-4252-9c88-706b57a783ce None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:51.176917 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:51.194943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b159-41d5-4c5d-8755-61d001a615f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:51.195569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b159-41d5-4c5d-8755-61d001a615f3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:51.195837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b159-41d5-4c5d-8755-61d001a615f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:51.196104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3921b159-41d5-4c5d-8755-61d001a615f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:51.207541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921b159-41d5-4c5d-8755-61d001a615f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:51.208927 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3921b159-41d5-4c5d-8755-61d001a615f3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-3921b159-41d5-4c5d-8755-61d001a615f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:51.227172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:51.228193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:51.229697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:51.229697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:51.229697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:51.229697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 379/762] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:51.234298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c157f17-c9a4-4a28-9348-d921c3c61491 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:51.370634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c157f17-c9a4-4a28-9348-d921c3c61491 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:51.401975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:51.412554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4e6c0-fc13-4892-93fb-e584595c5166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:51.413148 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4e6c0-fc13-4892-93fb-e584595c5166 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:51.413418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4e6c0-fc13-4892-93fb-e584595c5166 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:51.413740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4e6c0-fc13-4892-93fb-e584595c5166 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:51.414418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4e6c0-fc13-4892-93fb-e584595c5166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:51.414713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4e6c0-fc13-4892-93fb-e584595c5166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:51.416071 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4e6c0-fc13-4892-93fb-e584595c5166 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:51.428307 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 384/763] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:51.434121 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b159-41d5-4c5d-8755-61d001a615f3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:51.569697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3921b159-41d5-4c5d-8755-61d001a615f3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:51.609395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed384cae-ceeb-42a3-9bb3-1404167579a6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:51.609855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed384cae-ceeb-42a3-9bb3-1404167579a6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:51.610035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed384cae-ceeb-42a3-9bb3-1404167579a6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:51.610735 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f565a-d6b6-452f-82f1-8abf1e5363ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:51.611363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f565a-d6b6-452f-82f1-8abf1e5363ae None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:51.611630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 385/764] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:51.611798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f565a-d6b6-452f-82f1-8abf1e5363ae None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:51.611947 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f565a-d6b6-452f-82f1-8abf1e5363ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:51.616401 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b67ee7-7ff3-48b6-8996-5fa123effbe0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:51.616680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b67ee7-7ff3-48b6-8996-5fa123effbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:51.616965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b67ee7-7ff3-48b6-8996-5fa123effbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:51.623314 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f565a-d6b6-452f-82f1-8abf1e5363ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:51.624941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f565a-d6b6-452f-82f1-8abf1e5363ae None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:51.635554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a4b67ee7-7ff3-48b6-8996-5fa123effbe0 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:51.636737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a4b67ee7-7ff3-48b6-8996-5fa123effbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-953f565a-d6b6-452f-82f1-8abf1e5363ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:51.643134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:51.645291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:51.647476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:51.647476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:51.647476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:51.647476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:51.647476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:51.647476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:51.647476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:51.647476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:51.647476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 380/765] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:51.650279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f565a-d6b6-452f-82f1-8abf1e5363ae None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:51.695757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:51.743862 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 386/766] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:11:51.745141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5cfbc-ff0d-44a3-a336-7495da415040 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:51.745416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5cfbc-ff0d-44a3-a336-7495da415040 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:51.745680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5cfbc-ff0d-44a3-a336-7495da415040 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:51.747213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 387/767] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:51.768335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e53d34-cb22-4e69-870a-90da362cb8f8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:51.768636 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e53d34-cb22-4e69-870a-90da362cb8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:51.768861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e53d34-cb22-4e69-870a-90da362cb8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:51.807411 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b8e53d34-cb22-4e69-870a-90da362cb8f8 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:51.809007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b8e53d34-cb22-4e69-870a-90da362cb8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:51.909454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 388/768] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:11:51.936575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f565a-d6b6-452f-82f1-8abf1e5363ae None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:51.984083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c1a2e-7ee4-4d8d-86c0-ad6c9a3c2fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:51.985207 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c1a2e-7ee4-4d8d-86c0-ad6c9a3c2fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:51.985481 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c1a2e-7ee4-4d8d-86c0-ad6c9a3c2fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:51.985746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c1a2e-7ee4-4d8d-86c0-ad6c9a3c2fb0 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:51.986460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c1a2e-7ee4-4d8d-86c0-ad6c9a3c2fb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:51.986759 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c1a2e-7ee4-4d8d-86c0-ad6c9a3c2fb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:51.988173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c1a2e-7ee4-4d8d-86c0-ad6c9a3c2fb0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:52.006989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 381/769] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:52.012455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e53d34-cb22-4e69-870a-90da362cb8f8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:52.086427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c1a2e-7ee4-4d8d-86c0-ad6c9a3c2fb0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:52.099105 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:52.193647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e53d34-cb22-4e69-870a-90da362cb8f8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:52.222563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:52.247994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c1a2e-7ee4-4d8d-86c0-ad6c9a3c2fb0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:52.269968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab363e-fdf1-4a3d-9137-586964ec627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:52.269968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab363e-fdf1-4a3d-9137-586964ec627d None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:52.269968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab363e-fdf1-4a3d-9137-586964ec627d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:52.269968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab363e-fdf1-4a3d-9137-586964ec627d None admin] PATH_INFO: `/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/5537185f42474cfd86fca70fa8beddd2/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:52.272468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab363e-fdf1-4a3d-9137-586964ec627d None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1f2574a4344711a56722bca143ae6c, user_id=5537185f42474cfd86fca70fa8beddd2, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:52.274037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab363e-fdf1-4a3d-9137-586964ec627d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:52.293003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746574e-ff46-4c73-abc8-5bdcabc70929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:52.293638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746574e-ff46-4c73-abc8-5bdcabc70929 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:52.293907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746574e-ff46-4c73-abc8-5bdcabc70929 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:52.294198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746574e-ff46-4c73-abc8-5bdcabc70929 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:52.302683 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 389/770] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:11:52 2022] PUT /identity/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/5537185f42474cfd86fca70fa8beddd2/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:11:52.304274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab363e-fdf1-4a3d-9137-586964ec627d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:52.305558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746574e-ff46-4c73-abc8-5bdcabc70929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-e746574e-ff46-4c73-abc8-5bdcabc70929 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:52.310641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:52.311578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:52.312620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:52.312620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:52.312620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:52.312620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:52.312620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:52.312620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:52.312620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:52.312620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:52.312620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:52.312620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:52.312620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:52.312620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:52.312620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 382/771] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:52.322413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746574e-ff46-4c73-abc8-5bdcabc70929 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:52.433468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746574e-ff46-4c73-abc8-5bdcabc70929 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:52.475872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bd4b2-748e-4b66-8c73-f91923d9801b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:52.476557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bd4b2-748e-4b66-8c73-f91923d9801b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:52.476937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bd4b2-748e-4b66-8c73-f91923d9801b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:52.477323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bd4b2-748e-4b66-8c73-f91923d9801b None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:52.478237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bd4b2-748e-4b66-8c73-f91923d9801b None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:52.478640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bd4b2-748e-4b66-8c73-f91923d9801b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:52.480208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bd4b2-748e-4b66-8c73-f91923d9801b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:52.490461 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 383/772] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:52.495899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bd4b2-748e-4b66-8c73-f91923d9801b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:52.521005 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:52.528452 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab363e-fdf1-4a3d-9137-586964ec627d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:52.539979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:52.576895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f603fc-616d-49e0-8f08-41f5b8b748ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:52.577536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f603fc-616d-49e0-8f08-41f5b8b748ab None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:52.577809 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f603fc-616d-49e0-8f08-41f5b8b748ab None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:52.578092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f603fc-616d-49e0-8f08-41f5b8b748ab None admin] PATH_INFO: `/v3/services/network` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:52.578772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f603fc-616d-49e0-8f08-41f5b8b748ab None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:52.580381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f603fc-616d-49e0-8f08-41f5b8b748ab None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-05f603fc-616d-49e0-8f08-41f5b8b748ab None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:52.593807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:52.595539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 17:11:52.596415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:52.597167 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 390/773] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:52 2022] GET /identity/v3/services/network => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:52.607386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f603fc-616d-49e0-8f08-41f5b8b748ab None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:52.655509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bd4b2-748e-4b66-8c73-f91923d9801b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:52.697838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21438c6b-217e-45d0-a9e2-ab86e6b50296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:52.698513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21438c6b-217e-45d0-a9e2-ab86e6b50296 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:52.698786 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21438c6b-217e-45d0-a9e2-ab86e6b50296 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:52.699054 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21438c6b-217e-45d0-a9e2-ab86e6b50296 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:52.711903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21438c6b-217e-45d0-a9e2-ab86e6b50296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:52.713345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21438c6b-217e-45d0-a9e2-ab86e6b50296 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-21438c6b-217e-45d0-a9e2-ab86e6b50296 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:52.728554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:52.729509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:52.730694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:52.730694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:52.730694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:52.730694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 384/774] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:52.731360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60775c-9c1c-4dfd-b96d-7a90a296f002 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:52.734091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60775c-9c1c-4dfd-b96d-7a90a296f002 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:52.734091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60775c-9c1c-4dfd-b96d-7a90a296f002 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:52.734091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 385/775] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:52.735879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60775c-9c1c-4dfd-b96d-7a90a296f002 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:52.746459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f603fc-616d-49e0-8f08-41f5b8b748ab None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:52.785497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebc117-cc38-45e3-b047-9e85faf893f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:52.786175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebc117-cc38-45e3-b047-9e85faf893f6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:52.786455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebc117-cc38-45e3-b047-9e85faf893f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:52.786783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebc117-cc38-45e3-b047-9e85faf893f6 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:52.787437 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebc117-cc38-45e3-b047-9e85faf893f6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:52.787734 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebc117-cc38-45e3-b047-9e85faf893f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:52.789212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebc117-cc38-45e3-b047-9e85faf893f6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:52.794944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 391/776] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Sep 27 17:11:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:52.796279 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f525dbb-016d-4c58-9cb8-f20db3a1bf66 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:52.796607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f525dbb-016d-4c58-9cb8-f20db3a1bf66 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:52.796882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f525dbb-016d-4c58-9cb8-f20db3a1bf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:52.819087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9f525dbb-016d-4c58-9cb8-f20db3a1bf66 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:52.820268 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9f525dbb-016d-4c58-9cb8-f20db3a1bf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:52.821614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:52.850725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:52.876509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60775c-9c1c-4dfd-b96d-7a90a296f002 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:52.905159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 392/777] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:11:52.906297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f525dbb-016d-4c58-9cb8-f20db3a1bf66 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:52.912853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bbc58-2134-40d0-b691-092cd09d3269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:52.913539 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bbc58-2134-40d0-b691-092cd09d3269 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:52.913881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bbc58-2134-40d0-b691-092cd09d3269 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:52.914256 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bbc58-2134-40d0-b691-092cd09d3269 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:52.915074 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bbc58-2134-40d0-b691-092cd09d3269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:52.915444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bbc58-2134-40d0-b691-092cd09d3269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:52.916883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bbc58-2134-40d0-b691-092cd09d3269 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:52.930956 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 386/778] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:52.932591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dadecd-4fbc-4ee4-a25d-be61defdc4be None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:52.932939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dadecd-4fbc-4ee4-a25d-be61defdc4be None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:52.933351 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dadecd-4fbc-4ee4-a25d-be61defdc4be None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:52.935088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 387/779] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:52.951128 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dadecd-4fbc-4ee4-a25d-be61defdc4be None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:53.062100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f525dbb-016d-4c58-9cb8-f20db3a1bf66 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:53.063297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dadecd-4fbc-4ee4-a25d-be61defdc4be None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:53.086381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:53.100082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04093618-2e35-4900-99e1-a3f4eb24b3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:53.100824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04093618-2e35-4900-99e1-a3f4eb24b3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:53.101198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04093618-2e35-4900-99e1-a3f4eb24b3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:53.101586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04093618-2e35-4900-99e1-a3f4eb24b3d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:53.109316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04093618-2e35-4900-99e1-a3f4eb24b3d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:53.110841 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099e324-ba86-4620-ac52-d2041719a429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:53.111503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099e324-ba86-4620-ac52-d2041719a429 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:53.111721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04093618-2e35-4900-99e1-a3f4eb24b3d8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:53.111828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099e324-ba86-4620-ac52-d2041719a429 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:53.112050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099e324-ba86-4620-ac52-d2041719a429 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:53.112743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099e324-ba86-4620-ac52-d2041719a429 None admin] RBAC: Adding query filter params (type=network) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:53.113046 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099e324-ba86-4620-ac52-d2041719a429 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:53.115298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a099e324-ba86-4620-ac52-d2041719a429 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-04093618-2e35-4900-99e1-a3f4eb24b3d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:53.124413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:53.125694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:53.127542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:53.127542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:53.127542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:53.127542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:53.127542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:53.127542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:53.127542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:53.127542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:53.127542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 17:11:53.127542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:53.128810 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 393/780] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Sep 27 17:11:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:53.128810 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64debc65-f11c-4dac-956f-255acc21dee2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:53.129033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 388/781] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Sep 27 17:11:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:53.129127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64debc65-f11c-4dac-956f-255acc21dee2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:53.129127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64debc65-f11c-4dac-956f-255acc21dee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:53.129371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04093618-2e35-4900-99e1-a3f4eb24b3d8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:53.152237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-64debc65-f11c-4dac-956f-255acc21dee2 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:53.153612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-64debc65-f11c-4dac-956f-255acc21dee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:53.178856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:53.225935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 394/782] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:11:53.226925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64debc65-f11c-4dac-956f-255acc21dee2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:53.251330 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04093618-2e35-4900-99e1-a3f4eb24b3d8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:53.295005 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f1948-5890-40b4-839e-e2255a91349a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:53.296008 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f1948-5890-40b4-839e-e2255a91349a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:53.296615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f1948-5890-40b4-839e-e2255a91349a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:53.297670 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f1948-5890-40b4-839e-e2255a91349a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:53.298680 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f1948-5890-40b4-839e-e2255a91349a None admin] RBAC: Adding query filter params (interface=public, service_id=6eb1f93a20a644f8b009a0f7c65d1349, region_id=RegionOne) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:53.299209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f1948-5890-40b4-839e-e2255a91349a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:53.300964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f1948-5890-40b4-839e-e2255a91349a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:53.310088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 389/783] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Sep 27 17:11:53 2022] GET /identity/v3/endpoints?service_id=6eb1f93a20a644f8b009a0f7c65d1349&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:53.311360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f1948-5890-40b4-839e-e2255a91349a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:53.333792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64debc65-f11c-4dac-956f-255acc21dee2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:53.368841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:53.369020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83988a91-5993-4f7f-9918-0bca2d1aebf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:53.369471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83988a91-5993-4f7f-9918-0bca2d1aebf7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:53.369745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83988a91-5993-4f7f-9918-0bca2d1aebf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:53.370422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83988a91-5993-4f7f-9918-0bca2d1aebf7 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:53.371161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83988a91-5993-4f7f-9918-0bca2d1aebf7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:53.371466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83988a91-5993-4f7f-9918-0bca2d1aebf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:53.372889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83988a91-5993-4f7f-9918-0bca2d1aebf7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:53.397348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 395/784] 10.222.0.55 () {56 vars in 1149 bytes} [Tue Sep 27 17:11:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:53.397348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83988a91-5993-4f7f-9918-0bca2d1aebf7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:53.416153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:53.440373 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f1948-5890-40b4-839e-e2255a91349a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:53.476740 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54637019-4fee-4136-a49a-1dcd95b045a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:53.477581 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54637019-4fee-4136-a49a-1dcd95b045a1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:53.478033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54637019-4fee-4136-a49a-1dcd95b045a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:53.478552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54637019-4fee-4136-a49a-1dcd95b045a1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:53.479282 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54637019-4fee-4136-a49a-1dcd95b045a1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:53.481156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54637019-4fee-4136-a49a-1dcd95b045a1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-54637019-4fee-4136-a49a-1dcd95b045a1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:53.491295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:53.492278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 17:11:53.493133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:53.501082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 390/785] 10.222.0.55 () {56 vars in 1129 bytes} [Tue Sep 27 17:11:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:53.502118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54637019-4fee-4136-a49a-1dcd95b045a1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:53.503804 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83988a91-5993-4f7f-9918-0bca2d1aebf7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:53.538353 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad39f9-dec8-4cba-a93f-4c53f7c93989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:53.539105 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad39f9-dec8-4cba-a93f-4c53f7c93989 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:53.539384 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad39f9-dec8-4cba-a93f-4c53f7c93989 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:53.539652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad39f9-dec8-4cba-a93f-4c53f7c93989 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:53.540255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad39f9-dec8-4cba-a93f-4c53f7c93989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:53.541601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad39f9-dec8-4cba-a93f-4c53f7c93989 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:53.550178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 396/786] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Sep 27 17:11:53 2022] GET /identity/v3/services => generated 772 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:53.550178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad39f9-dec8-4cba-a93f-4c53f7c93989 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:53.626342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54637019-4fee-4136-a49a-1dcd95b045a1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:53.639068 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:53.667494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5e22d-fb53-4f8f-b5b6-7324022375b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:53.668139 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5e22d-fb53-4f8f-b5b6-7324022375b0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:53.668414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5e22d-fb53-4f8f-b5b6-7324022375b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:53.668752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5e22d-fb53-4f8f-b5b6-7324022375b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:53.673884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:53.684480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5e22d-fb53-4f8f-b5b6-7324022375b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:53.686329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5e22d-fb53-4f8f-b5b6-7324022375b0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:53.699752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad39f9-dec8-4cba-a93f-4c53f7c93989 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-efe5e22d-fb53-4f8f-b5b6-7324022375b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:53.704291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:53.705521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:53.706691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:53.706691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:53.706691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:53.713133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 391/787] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:53.714454 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5e22d-fb53-4f8f-b5b6-7324022375b0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:53.743981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aef6a4-d15d-4871-8646-85883797c681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:53.744629 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aef6a4-d15d-4871-8646-85883797c681 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:53.744910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aef6a4-d15d-4871-8646-85883797c681 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:53.745183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aef6a4-d15d-4871-8646-85883797c681 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:53.745818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aef6a4-d15d-4871-8646-85883797c681 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:53.746276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aef6a4-d15d-4871-8646-85883797c681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:53.747667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aef6a4-d15d-4871-8646-85883797c681 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:53.754096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 397/788] 10.222.0.55 () {56 vars in 1124 bytes} [Tue Sep 27 17:11:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:53.755344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aef6a4-d15d-4871-8646-85883797c681 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:53.839064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5e22d-fb53-4f8f-b5b6-7324022375b0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:53.882654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585b9ba-125d-4b85-90c8-84be3f245e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:53.883113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585b9ba-125d-4b85-90c8-84be3f245e30 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:53.883400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585b9ba-125d-4b85-90c8-84be3f245e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:53.883675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585b9ba-125d-4b85-90c8-84be3f245e30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:53.894446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585b9ba-125d-4b85-90c8-84be3f245e30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:53.897364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aef6a4-d15d-4871-8646-85883797c681 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-f585b9ba-125d-4b85-90c8-84be3f245e30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:53.899678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:53.900881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:53.902009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:53.902009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:53.902009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:53.902009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:53.902009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:53.902009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:53.902009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:53.902009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:53.902009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:53.902009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:53.902009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:53.902009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:53.905674 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 392/789] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:53.907084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585b9ba-125d-4b85-90c8-84be3f245e30 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:53.933294 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:53.945416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b4c9c-ee86-4254-b805-8a2a28b7233c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:53.946355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b4c9c-ee86-4254-b805-8a2a28b7233c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:53.946780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b4c9c-ee86-4254-b805-8a2a28b7233c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:53.947197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b4c9c-ee86-4254-b805-8a2a28b7233c None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:53.948027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b4c9c-ee86-4254-b805-8a2a28b7233c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:53.948472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b4c9c-ee86-4254-b805-8a2a28b7233c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:53.949990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b4c9c-ee86-4254-b805-8a2a28b7233c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:53.968336 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 398/790] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:53.970556 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b4c9c-ee86-4254-b805-8a2a28b7233c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:53.974567 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:54.018149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585b9ba-125d-4b85-90c8-84be3f245e30 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:54.056942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e40fb-6aa4-4559-a7e5-484e31567022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:54.057390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e40fb-6aa4-4559-a7e5-484e31567022 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:54.057669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e40fb-6aa4-4559-a7e5-484e31567022 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:54.057940 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e40fb-6aa4-4559-a7e5-484e31567022 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:54.058749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e40fb-6aa4-4559-a7e5-484e31567022 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:54.058993 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e40fb-6aa4-4559-a7e5-484e31567022 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:54.060362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e40fb-6aa4-4559-a7e5-484e31567022 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:54.066376 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 393/791] 10.222.0.55 () {56 vars in 1124 bytes} [Tue Sep 27 17:11:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:54.071167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e40fb-6aa4-4559-a7e5-484e31567022 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:54.100091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b4c9c-ee86-4254-b805-8a2a28b7233c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:54.141335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47505505-2959-4ad4-a3ff-166c69ec19d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:54.141866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47505505-2959-4ad4-a3ff-166c69ec19d8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:54.142175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47505505-2959-4ad4-a3ff-166c69ec19d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:54.142448 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47505505-2959-4ad4-a3ff-166c69ec19d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:54.143116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47505505-2959-4ad4-a3ff-166c69ec19d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:54.143417 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47505505-2959-4ad4-a3ff-166c69ec19d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:54.144776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47505505-2959-4ad4-a3ff-166c69ec19d8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:54.158229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 399/792] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:54.160399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47505505-2959-4ad4-a3ff-166c69ec19d8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:54.183419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:54.207937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e40fb-6aa4-4559-a7e5-484e31567022 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:54.244487 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4c278-3f82-4e0a-a44c-5ef8c90e854c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:54.245120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4c278-3f82-4e0a-a44c-5ef8c90e854c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:54.245400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4c278-3f82-4e0a-a44c-5ef8c90e854c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:54.245668 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4c278-3f82-4e0a-a44c-5ef8c90e854c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:54.252888 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4c278-3f82-4e0a-a44c-5ef8c90e854c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:54.254358 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4c278-3f82-4e0a-a44c-5ef8c90e854c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:54.254684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-34b4c278-3f82-4e0a-a44c-5ef8c90e854c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:54.267377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:54.268311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:54.269218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:54.269218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:54.269218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:54.269218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:54.269218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:54.269218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:54.269218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:54.269218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:54.273486 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 394/793] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:54.274368 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4c278-3f82-4e0a-a44c-5ef8c90e854c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:54.304589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47505505-2959-4ad4-a3ff-166c69ec19d8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:54.354212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d5a08-f4d9-4754-a89e-138a4cd6ff85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:54.356994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d5a08-f4d9-4754-a89e-138a4cd6ff85 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:54.357504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d5a08-f4d9-4754-a89e-138a4cd6ff85 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:54.357979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d5a08-f4d9-4754-a89e-138a4cd6ff85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:54.358979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d5a08-f4d9-4754-a89e-138a4cd6ff85 None admin] RBAC: Adding query filter params (interface=public, service_id=0f4019e1fd13498d92f4b8a86da133dd, region_id=RegionOne) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:54.360075 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d5a08-f4d9-4754-a89e-138a4cd6ff85 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:54.361707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d5a08-f4d9-4754-a89e-138a4cd6ff85 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:54.372833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 400/794] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Sep 27 17:11:54 2022] GET /identity/v3/endpoints?service_id=0f4019e1fd13498d92f4b8a86da133dd&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:54.374024 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d5a08-f4d9-4754-a89e-138a4cd6ff85 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:54.392548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4c278-3f82-4e0a-a44c-5ef8c90e854c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:54.427521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ce98f-d5d6-476f-ac4d-6d868b4a8715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:54.428156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ce98f-d5d6-476f-ac4d-6d868b4a8715 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:54.428434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ce98f-d5d6-476f-ac4d-6d868b4a8715 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:54.428706 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ce98f-d5d6-476f-ac4d-6d868b4a8715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:54.439869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ce98f-d5d6-476f-ac4d-6d868b4a8715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:54.441930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ce98f-d5d6-476f-ac4d-6d868b4a8715 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:54.448665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-697ce98f-d5d6-476f-ac4d-6d868b4a8715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:54.458608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:54.459594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:54.460515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:54.460515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:54.460515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:54.464470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 395/795] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:54.465305 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ce98f-d5d6-476f-ac4d-6d868b4a8715 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:54.512838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d5a08-f4d9-4754-a89e-138a4cd6ff85 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:54.551302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:54.560064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b23fee-b745-4b13-bab3-1a27bf0ced6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:54.561720 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b23fee-b745-4b13-bab3-1a27bf0ced6b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:54.562264 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b23fee-b745-4b13-bab3-1a27bf0ced6b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:54.563229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b23fee-b745-4b13-bab3-1a27bf0ced6b None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:54.564439 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b23fee-b745-4b13-bab3-1a27bf0ced6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:54.564932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b23fee-b745-4b13-bab3-1a27bf0ced6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:54.567762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b23fee-b745-4b13-bab3-1a27bf0ced6b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:54.576536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ce98f-d5d6-476f-ac4d-6d868b4a8715 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:54.591926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 401/796] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:54.593324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b23fee-b745-4b13-bab3-1a27bf0ced6b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:54.634794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e37548-2148-4c9f-be96-d5b473f38e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:54.636102 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e37548-2148-4c9f-be96-d5b473f38e4d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:54.636675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e37548-2148-4c9f-be96-d5b473f38e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:54.637201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e37548-2148-4c9f-be96-d5b473f38e4d None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:54.638127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e37548-2148-4c9f-be96-d5b473f38e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:54.641738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e37548-2148-4c9f-be96-d5b473f38e4d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:54.649214 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 396/797] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Sep 27 17:11:54 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:54.650317 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e37548-2148-4c9f-be96-d5b473f38e4d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:54.725739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:54.739457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b23fee-b745-4b13-bab3-1a27bf0ced6b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:54.779674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa4449-741b-4327-a620-0b9474f5cbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:54.780762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa4449-741b-4327-a620-0b9474f5cbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:54.781357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa4449-741b-4327-a620-0b9474f5cbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:54.781825 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa4449-741b-4327-a620-0b9474f5cbc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:54.782593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e37548-2148-4c9f-be96-d5b473f38e4d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:54.782936 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa4449-741b-4327-a620-0b9474f5cbc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:54.783428 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa4449-741b-4327-a620-0b9474f5cbc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:54.786381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa4449-741b-4327-a620-0b9474f5cbc2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:54.806034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 402/798] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:54.808320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d460b2e-71ca-4a5f-ade2-f02c33bd5ff0 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:54.808853 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d460b2e-71ca-4a5f-ade2-f02c33bd5ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:54.809320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d460b2e-71ca-4a5f-ade2-f02c33bd5ff0 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:54.812073 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 403/799] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:54.813683 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d460b2e-71ca-4a5f-ade2-f02c33bd5ff0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:54.834865 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd8e3e-baa9-4f6b-bc62-d84eb49c7f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:54.835848 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd8e3e-baa9-4f6b-bc62-d84eb49c7f72 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:54.836130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd8e3e-baa9-4f6b-bc62-d84eb49c7f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:54.836404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd8e3e-baa9-4f6b-bc62-d84eb49c7f72 None admin] PATH_INFO: `/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/c8c3fcd3789c40c88a4c32fff2678fc9/roles/fafa3faaeb1344d2b6f639709d49e59b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:54.860239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd8e3e-baa9-4f6b-bc62-d84eb49c7f72 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1f2574a4344711a56722bca143ae6c, user_id=c8c3fcd3789c40c88a4c32fff2678fc9, role_id=fafa3faaeb1344d2b6f639709d49e59b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:54.861692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bd8e3e-baa9-4f6b-bc62-d84eb49c7f72 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:54.880057 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 397/800] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:11:54 2022] PUT /identity/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/c8c3fcd3789c40c88a4c32fff2678fc9/roles/fafa3faaeb1344d2b6f639709d49e59b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:11:54.881081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5b08d-d6df-444a-9233-418aca586040 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:54.881366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5b08d-d6df-444a-9233-418aca586040 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:54.881629 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5b08d-d6df-444a-9233-418aca586040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:54.885392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:54.902833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-43f5b08d-d6df-444a-9233-418aca586040 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:54.904484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-43f5b08d-d6df-444a-9233-418aca586040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:54.955204 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d460b2e-71ca-4a5f-ade2-f02c33bd5ff0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:54.989776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 398/801] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:11:55.000576 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141aab1-e997-4082-b355-0afa45cf1447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:55.001626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141aab1-e997-4082-b355-0afa45cf1447 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:55.002130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141aab1-e997-4082-b355-0afa45cf1447 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:55.002623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c141aab1-e997-4082-b355-0afa45cf1447 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:55.132371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141aab1-e997-4082-b355-0afa45cf1447 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:55.133837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c141aab1-e997-4082-b355-0afa45cf1447 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c141aab1-e997-4082-b355-0afa45cf1447 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:55.146695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:55.147717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:55.149074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:55.149074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:55.149074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:55.149074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:55.149074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:55.149074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:55.149074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:55.149074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:55.149074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 27 17:11:55.149074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:55.149074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 404/802] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:55.153758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5b08d-d6df-444a-9233-418aca586040 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:55.215641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:55.282647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5b08d-d6df-444a-9233-418aca586040 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:55.326093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417af3d-75e8-49e4-9ccf-f7b295bd672d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:55.326937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417af3d-75e8-49e4-9ccf-f7b295bd672d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:55.327354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417af3d-75e8-49e4-9ccf-f7b295bd672d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:55.327761 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417af3d-75e8-49e4-9ccf-f7b295bd672d None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:55.328704 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417af3d-75e8-49e4-9ccf-f7b295bd672d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:55.329146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417af3d-75e8-49e4-9ccf-f7b295bd672d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:55.338095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417af3d-75e8-49e4-9ccf-f7b295bd672d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:55.344847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141aab1-e997-4082-b355-0afa45cf1447 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:55.369889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 399/803] 10.222.0.55 () {56 vars in 1135 bytes} [Tue Sep 27 17:11:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:55.375245 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417af3d-75e8-49e4-9ccf-f7b295bd672d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:55.535483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:55.540579 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417af3d-75e8-49e4-9ccf-f7b295bd672d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:55.544743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c141aab1-e997-4082-b355-0afa45cf1447 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:55.564704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:55.609620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c00528-32f2-4861-bf55-2de9bebcc491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:55.610284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c00528-32f2-4861-bf55-2de9bebcc491 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:55.610561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c00528-32f2-4861-bf55-2de9bebcc491 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:55.610828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c00528-32f2-4861-bf55-2de9bebcc491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:55.613257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb60169-d1e4-4312-b037-a18e9e2a6287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:55.613938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb60169-d1e4-4312-b037-a18e9e2a6287 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:55.614176 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb60169-d1e4-4312-b037-a18e9e2a6287 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:55.614445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb60169-d1e4-4312-b037-a18e9e2a6287 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:55.632507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c00528-32f2-4861-bf55-2de9bebcc491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:55.633975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c00528-32f2-4861-bf55-2de9bebcc491 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:55.638912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb60169-d1e4-4312-b037-a18e9e2a6287 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-fbb60169-d1e4-4312-b037-a18e9e2a6287 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:55.644338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:55.645294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:55.646157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:55.646157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:55.646157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:55.646157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:55.646157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:55.646157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:55.646157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:55.646157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:55.646157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:55.646157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:55.646157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:55.646157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:55.654261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 405/804] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:55.656201 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb60169-d1e4-4312-b037-a18e9e2a6287 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-e5c00528-32f2-4861-bf55-2de9bebcc491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:55.664525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:55.665496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:55.666430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:55.666430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:55.666430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:55.666430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 400/805] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:55.674535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c00528-32f2-4861-bf55-2de9bebcc491 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:55.847707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb60169-d1e4-4312-b037-a18e9e2a6287 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:55.848374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c00528-32f2-4861-bf55-2de9bebcc491 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:55.886242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb05be0-8a12-4c92-816f-d498b220b821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:55.886840 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb05be0-8a12-4c92-816f-d498b220b821 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:55.887110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb05be0-8a12-4c92-816f-d498b220b821 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:55.887324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446e433-46f2-47ac-8dd5-7cdfbb43289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:55.887442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb05be0-8a12-4c92-816f-d498b220b821 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:55.887938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446e433-46f2-47ac-8dd5-7cdfbb43289d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:55.888053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb05be0-8a12-4c92-816f-d498b220b821 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:55.888207 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446e433-46f2-47ac-8dd5-7cdfbb43289d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:55.888297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb05be0-8a12-4c92-816f-d498b220b821 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:55.888481 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446e433-46f2-47ac-8dd5-7cdfbb43289d None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:55.889131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446e433-46f2-47ac-8dd5-7cdfbb43289d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:55.889433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446e433-46f2-47ac-8dd5-7cdfbb43289d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:55.889619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb05be0-8a12-4c92-816f-d498b220b821 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:55.891446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446e433-46f2-47ac-8dd5-7cdfbb43289d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:55.899701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 406/806] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:55.904685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 401/807] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:55.904830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e784ef-0d73-42ad-99cc-69d9e05b05b5 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:55.905117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e784ef-0d73-42ad-99cc-69d9e05b05b5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:55.905499 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e784ef-0d73-42ad-99cc-69d9e05b05b5 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:55.910157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 407/808] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:55.912543 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e784ef-0d73-42ad-99cc-69d9e05b05b5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:55.912672 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446e433-46f2-47ac-8dd5-7cdfbb43289d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:55.928489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:55.947370 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:56.023346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e784ef-0d73-42ad-99cc-69d9e05b05b5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:56.024408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446e433-46f2-47ac-8dd5-7cdfbb43289d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:56.058395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830fc08-f69a-4604-8a44-ec902c910329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:56.059037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830fc08-f69a-4604-8a44-ec902c910329 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:56.059306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830fc08-f69a-4604-8a44-ec902c910329 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:56.059570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830fc08-f69a-4604-8a44-ec902c910329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:56.062168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71b74f-e5be-4ec1-a1cd-31f50a7310fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:56.063481 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71b74f-e5be-4ec1-a1cd-31f50a7310fa None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:56.063762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71b74f-e5be-4ec1-a1cd-31f50a7310fa None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:56.064031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71b74f-e5be-4ec1-a1cd-31f50a7310fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:56.072195 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71b74f-e5be-4ec1-a1cd-31f50a7310fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:56.072495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830fc08-f69a-4604-8a44-ec902c910329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:56.073647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830fc08-f69a-4604-8a44-ec902c910329 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:56.073825 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71b74f-e5be-4ec1-a1cd-31f50a7310fa None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-7f71b74f-e5be-4ec1-a1cd-31f50a7310fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:56.086179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:56.088202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:56.090354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:56.090354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:56.090354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:56.090354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:56.090354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:56.090354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:56.090354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:56.090354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:56.090354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 402/809] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-5830fc08-f69a-4604-8a44-ec902c910329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:56.091475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:56.093504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:56.103338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:56.103338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:56.103338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:56.103583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111757c-9544-42ac-a9c0-23b6445b3c02 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:56.103583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111757c-9544-42ac-a9c0-23b6445b3c02 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:56.103583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111757c-9544-42ac-a9c0-23b6445b3c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:56.103804 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 408/810] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:56.103804 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830fc08-f69a-4604-8a44-ec902c910329 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:56.116294 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-4111757c-9544-42ac-a9c0-23b6445b3c02 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:56.117548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4111757c-9544-42ac-a9c0-23b6445b3c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:56.190213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 403/811] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:11:56.191096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111757c-9544-42ac-a9c0-23b6445b3c02 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:56.192732 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:56.194142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:56.211511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830fc08-f69a-4604-8a44-ec902c910329 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:56.262151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872df40b-9aff-4f7c-a164-1dc369a337cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:56.262753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872df40b-9aff-4f7c-a164-1dc369a337cf None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:56.263163 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872df40b-9aff-4f7c-a164-1dc369a337cf None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:56.263483 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872df40b-9aff-4f7c-a164-1dc369a337cf None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:56.264272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872df40b-9aff-4f7c-a164-1dc369a337cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:56.264571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872df40b-9aff-4f7c-a164-1dc369a337cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:56.266016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872df40b-9aff-4f7c-a164-1dc369a337cf None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:56.292453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 409/812] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:56.299164 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872df40b-9aff-4f7c-a164-1dc369a337cf None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:56.316143 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111757c-9544-42ac-a9c0-23b6445b3c02 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:56.356749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d44c86-4a82-41b2-9998-144db1a1be84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:56.357987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d44c86-4a82-41b2-9998-144db1a1be84 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:56.358296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d44c86-4a82-41b2-9998-144db1a1be84 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:56.358568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d44c86-4a82-41b2-9998-144db1a1be84 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:56.359219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d44c86-4a82-41b2-9998-144db1a1be84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:56.360076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d44c86-4a82-41b2-9998-144db1a1be84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:56.361443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d44c86-4a82-41b2-9998-144db1a1be84 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:56.377492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 404/813] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:56.378896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d44c86-4a82-41b2-9998-144db1a1be84 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:56.442623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:56.447983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872df40b-9aff-4f7c-a164-1dc369a337cf None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:56.486170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb451e28-d56c-4cc0-ab75-2568f1b33f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:56.486817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb451e28-d56c-4cc0-ab75-2568f1b33f65 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:56.487088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb451e28-d56c-4cc0-ab75-2568f1b33f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:56.490104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb451e28-d56c-4cc0-ab75-2568f1b33f65 None admin] PATH_INFO: `/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/39073006a0054b56bb1bba46d2d0768e/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:56.491635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb451e28-d56c-4cc0-ab75-2568f1b33f65 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1f2574a4344711a56722bca143ae6c, user_id=39073006a0054b56bb1bba46d2d0768e, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:56.494299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d44c86-4a82-41b2-9998-144db1a1be84 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:56.496455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb451e28-d56c-4cc0-ab75-2568f1b33f65 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:56.518689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 410/814] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:11:56 2022] PUT /identity/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/39073006a0054b56bb1bba46d2d0768e/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:11:56.561403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad4f2f-f0c1-4e45-a6ab-4039fb23ad97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:56.562584 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad4f2f-f0c1-4e45-a6ab-4039fb23ad97 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:56.563062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad4f2f-f0c1-4e45-a6ab-4039fb23ad97 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:56.563529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad4f2f-f0c1-4e45-a6ab-4039fb23ad97 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:56.583498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad4f2f-f0c1-4e45-a6ab-4039fb23ad97 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:56.585410 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad4f2f-f0c1-4e45-a6ab-4039fb23ad97 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-60ad4f2f-f0c1-4e45-a6ab-4039fb23ad97 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:56.602109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:56.603169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:56.604120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:56.604120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:56.604120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:56.604120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:56.604120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:56.604120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:56.604120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:56.604120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:56.604120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:11:56.604120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:56.607163 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 405/815] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Sep 27 17:11:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:56.616630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb451e28-d56c-4cc0-ab75-2568f1b33f65 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:56.623467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:56.664494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad4f2f-f0c1-4e45-a6ab-4039fb23ad97 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:56.786110 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad4f2f-f0c1-4e45-a6ab-4039fb23ad97 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:56.811045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb451e28-d56c-4cc0-ab75-2568f1b33f65 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:56.821724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:56.838594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20868ef0-59f4-4745-9264-4b7d3be1f0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:56.839476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20868ef0-59f4-4745-9264-4b7d3be1f0cf None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:56.840169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20868ef0-59f4-4745-9264-4b7d3be1f0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:56.840444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20868ef0-59f4-4745-9264-4b7d3be1f0cf None admin] PATH_INFO: `/v3/services/network` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:56.841292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20868ef0-59f4-4745-9264-4b7d3be1f0cf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:56.843319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20868ef0-59f4-4745-9264-4b7d3be1f0cf None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:56.850714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc77d5f-6759-4c0b-8f21-db08d082c9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:56.851319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc77d5f-6759-4c0b-8f21-db08d082c9fc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:56.851592 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc77d5f-6759-4c0b-8f21-db08d082c9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:56.851859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc77d5f-6759-4c0b-8f21-db08d082c9fc None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:56.852586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc77d5f-6759-4c0b-8f21-db08d082c9fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:56.852886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc77d5f-6759-4c0b-8f21-db08d082c9fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-20868ef0-59f4-4745-9264-4b7d3be1f0cf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:56.854241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:56.855783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 27 17:11:56.856633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:56.857437 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc77d5f-6759-4c0b-8f21-db08d082c9fc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:56.857519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 406/816] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:56 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:56.862551 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20868ef0-59f4-4745-9264-4b7d3be1f0cf None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:56.899337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 411/817] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Sep 27 17:11:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:56.905667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc77d5f-6759-4c0b-8f21-db08d082c9fc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:57.010238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:57.050702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc77d5f-6759-4c0b-8f21-db08d082c9fc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:57.085903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dee23-d196-4d5a-9425-d8d8b2579bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:57.086538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dee23-d196-4d5a-9425-d8d8b2579bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:57.086815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dee23-d196-4d5a-9425-d8d8b2579bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:57.087078 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dee23-d196-4d5a-9425-d8d8b2579bc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:57.091987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20868ef0-59f4-4745-9264-4b7d3be1f0cf None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:57.159494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dee23-d196-4d5a-9425-d8d8b2579bc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:57.160914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9dee23-d196-4d5a-9425-d8d8b2579bc2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:57.168215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548097d-9259-4975-b465-c2521be61d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:57.168823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548097d-9259-4975-b465-c2521be61d5c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:57.169096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548097d-9259-4975-b465-c2521be61d5c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:57.169449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548097d-9259-4975-b465-c2521be61d5c None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:57.170001 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548097d-9259-4975-b465-c2521be61d5c None admin] RBAC: Adding query filter params (name=network) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:57.170333 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548097d-9259-4975-b465-c2521be61d5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:57.171696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548097d-9259-4975-b465-c2521be61d5c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:57.188430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 407/818] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Sep 27 17:11:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:57.204606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548097d-9259-4975-b465-c2521be61d5c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:57.229965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:57.279599 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 412/819] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:11:57.285571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dee23-d196-4d5a-9425-d8d8b2579bc2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:57.416840 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dee23-d196-4d5a-9425-d8d8b2579bc2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:57.422677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:57.433178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548097d-9259-4975-b465-c2521be61d5c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:57.458221 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1c3d2-4cd0-4d8a-a9ae-2f52e2cbbba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:57.459254 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1c3d2-4cd0-4d8a-a9ae-2f52e2cbbba5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:57.459772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1c3d2-4cd0-4d8a-a9ae-2f52e2cbbba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:57.460257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1c3d2-4cd0-4d8a-a9ae-2f52e2cbbba5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:57.472171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1c3d2-4cd0-4d8a-a9ae-2f52e2cbbba5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:57.473837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1c3d2-4cd0-4d8a-a9ae-2f52e2cbbba5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:57.475704 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41c2cf-5c48-40d2-9a00-4b8a31594e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:57.476307 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41c2cf-5c48-40d2-9a00-4b8a31594e7b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:57.476579 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41c2cf-5c48-40d2-9a00-4b8a31594e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:57.476841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41c2cf-5c48-40d2-9a00-4b8a31594e7b None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:57.477473 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41c2cf-5c48-40d2-9a00-4b8a31594e7b None admin] RBAC: Adding query filter params (type=network) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:57.477772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41c2cf-5c48-40d2-9a00-4b8a31594e7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:57.479143 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41c2cf-5c48-40d2-9a00-4b8a31594e7b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:57.484170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 408/820] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Sep 27 17:11:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:11:57.485345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4cec6-eb26-408f-aa1f-2e1467d5235c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:57.485647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4cec6-eb26-408f-aa1f-2e1467d5235c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:57.485882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4cec6-eb26-408f-aa1f-2e1467d5235c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:57.491127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 409/821] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:57.491127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb022e5-1691-4815-ad7d-b80e1e0392ed None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:57.491127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb022e5-1691-4815-ad7d-b80e1e0392ed None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:57.491127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb022e5-1691-4815-ad7d-b80e1e0392ed None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:57.491127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 410/822] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:11:57.491513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb022e5-1691-4815-ad7d-b80e1e0392ed None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-e2f1c3d2-4cd0-4d8a-a9ae-2f52e2cbbba5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:57.499569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:57.500570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:57.501459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:57.501459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:57.501459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:57.507378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 413/823] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:57.507378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9a103-e049-4d45-8450-e728b43e1b57 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:57.507378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9a103-e049-4d45-8450-e728b43e1b57 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:57.507378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9a103-e049-4d45-8450-e728b43e1b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:57.525334 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-26a9a103-e049-4d45-8450-e728b43e1b57 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:57.526548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-26a9a103-e049-4d45-8450-e728b43e1b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:57.537072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:57.594578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 414/824] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:11:57.596216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c969c-8e2d-4865-a559-344e9e1015db None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:57.596499 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c969c-8e2d-4865-a559-344e9e1015db None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:57.596769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c969c-8e2d-4865-a559-344e9e1015db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:57.616304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-238c969c-8e2d-4865-a559-344e9e1015db None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:57.617508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-238c969c-8e2d-4865-a559-344e9e1015db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:57.619966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb022e5-1691-4815-ad7d-b80e1e0392ed None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:57.637526 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:57.665261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6332f3e-c42e-4bc4-befc-c94975336a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:57.666260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6332f3e-c42e-4bc4-befc-c94975336a93 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:57.666730 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6332f3e-c42e-4bc4-befc-c94975336a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:57.667256 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6332f3e-c42e-4bc4-befc-c94975336a93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:57.668106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6332f3e-c42e-4bc4-befc-c94975336a93 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:57.670051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6332f3e-c42e-4bc4-befc-c94975336a93 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:57.705244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 415/825] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:11:57.706706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c969c-8e2d-4865-a559-344e9e1015db None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:57.711666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 411/826] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:11:57 2022] POST /identity/v3/endpoints => generated 337 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:11:57.800461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:57.831707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c969c-8e2d-4865-a559-344e9e1015db None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:57.875615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17415fbe-82f8-4175-bfd4-7881c7860233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:57.876632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17415fbe-82f8-4175-bfd4-7881c7860233 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:57.877119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17415fbe-82f8-4175-bfd4-7881c7860233 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:57.877580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17415fbe-82f8-4175-bfd4-7881c7860233 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:57.878456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17415fbe-82f8-4175-bfd4-7881c7860233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:57.878967 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17415fbe-82f8-4175-bfd4-7881c7860233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:57.880151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6332f3e-c42e-4bc4-befc-c94975336a93 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:57.883609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17415fbe-82f8-4175-bfd4-7881c7860233 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:57.905263 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 416/827] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:57.910880 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17415fbe-82f8-4175-bfd4-7881c7860233 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:58.036915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17415fbe-82f8-4175-bfd4-7881c7860233 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:58.073399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6332f3e-c42e-4bc4-befc-c94975336a93 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:58.075259 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:58.084673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd8067-adbc-4932-acab-125665f237a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:58.085308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd8067-adbc-4932-acab-125665f237a5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:58.086065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd8067-adbc-4932-acab-125665f237a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:58.086372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd8067-adbc-4932-acab-125665f237a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:58.096055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd8067-adbc-4932-acab-125665f237a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:58.097516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfd8067-adbc-4932-acab-125665f237a5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ecfd8067-adbc-4932-acab-125665f237a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:11:58.114237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:58.115153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:58.116037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:11:58.116037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:11:58.116037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:11:58.116037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:11:58.116037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:11:58.116037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:11:58.116037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:11:58.116037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:11:58.116037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 417/828] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:11:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:58.121123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd8067-adbc-4932-acab-125665f237a5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:58.122695 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf164e01-7d2b-4949-bded-125c18a2108b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:58.123337 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf164e01-7d2b-4949-bded-125c18a2108b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:58.123613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf164e01-7d2b-4949-bded-125c18a2108b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:58.123884 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf164e01-7d2b-4949-bded-125c18a2108b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:58.124801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf164e01-7d2b-4949-bded-125c18a2108b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:58.127792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf164e01-7d2b-4949-bded-125c18a2108b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-bf164e01-7d2b-4949-bded-125c18a2108b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:58.138694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:58.140238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:58.142635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 412/829] 10.222.0.55 () {56 vars in 1129 bytes} [Tue Sep 27 17:11:57 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:58.144833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf164e01-7d2b-4949-bded-125c18a2108b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:58.181060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:58.255974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd8067-adbc-4932-acab-125665f237a5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:58.258244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf164e01-7d2b-4949-bded-125c18a2108b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:58.290696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1cedb-b2cc-4e8c-800e-b8fae58bcb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:58.291301 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1cedb-b2cc-4e8c-800e-b8fae58bcb39 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:58.291646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1cedb-b2cc-4e8c-800e-b8fae58bcb39 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:58.291927 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1cedb-b2cc-4e8c-800e-b8fae58bcb39 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:58.292604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1cedb-b2cc-4e8c-800e-b8fae58bcb39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:58.292947 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1cedb-b2cc-4e8c-800e-b8fae58bcb39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:58.293529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883f55f-653e-4eb3-8809-ac3ff04efba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:58.294168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883f55f-653e-4eb3-8809-ac3ff04efba2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:58.294334 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1cedb-b2cc-4e8c-800e-b8fae58bcb39 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:58.294444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883f55f-653e-4eb3-8809-ac3ff04efba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:58.294698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883f55f-653e-4eb3-8809-ac3ff04efba2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:58.307903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883f55f-653e-4eb3-8809-ac3ff04efba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:58.311348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 418/830] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:11:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:58.312003 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1cedb-b2cc-4e8c-800e-b8fae58bcb39 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-4883f55f-653e-4eb3-8809-ac3ff04efba2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:58.314222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:58.315175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:58.316031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:58.316031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:58.316031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:58.316031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:58.316031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:58.316031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:58.316031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:58.316031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:58.316031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:58.316031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:58.316031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:11:58.316031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:58.320946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 413/831] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:11:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:58.321645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883f55f-653e-4eb3-8809-ac3ff04efba2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:58.369949 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:58.425959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:58.426438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1cedb-b2cc-4e8c-800e-b8fae58bcb39 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:58.433334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883f55f-653e-4eb3-8809-ac3ff04efba2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:58.460962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2b5be-fb6c-41a1-832b-ebb27b25d863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:58.461562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2b5be-fb6c-41a1-832b-ebb27b25d863 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:58.461837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2b5be-fb6c-41a1-832b-ebb27b25d863 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:58.462131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2b5be-fb6c-41a1-832b-ebb27b25d863 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:58.462762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2b5be-fb6c-41a1-832b-ebb27b25d863 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:58.463061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2b5be-fb6c-41a1-832b-ebb27b25d863 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:58.464421 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2b5be-fb6c-41a1-832b-ebb27b25d863 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:58.469378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 419/832] 10.222.0.55 () {56 vars in 1124 bytes} [Tue Sep 27 17:11:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:58.470252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2b5be-fb6c-41a1-832b-ebb27b25d863 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:58.476808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28687c54-6177-4831-ad0b-d20087febb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:58.477525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28687c54-6177-4831-ad0b-d20087febb79 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:58.477804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28687c54-6177-4831-ad0b-d20087febb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:58.478088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28687c54-6177-4831-ad0b-d20087febb79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:58.478836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28687c54-6177-4831-ad0b-d20087febb79 None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=5537185f42474cfd86fca70fa8beddd2) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:58.479140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28687c54-6177-4831-ad0b-d20087febb79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:58.480853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28687c54-6177-4831-ad0b-d20087febb79 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:58.494483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 414/833] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:11:58 2022] GET /identity/v3/role_assignments?user.id=5537185f42474cfd86fca70fa8beddd2&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:58.495796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28687c54-6177-4831-ad0b-d20087febb79 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:58.610998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28687c54-6177-4831-ad0b-d20087febb79 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:58.622841 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2b5be-fb6c-41a1-832b-ebb27b25d863 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:58.639431 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:58.666065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0b336-7e06-46ef-9889-89e17671a51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:58.666726 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0b336-7e06-46ef-9889-89e17671a51b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:58.667001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0b336-7e06-46ef-9889-89e17671a51b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:58.667271 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0b336-7e06-46ef-9889-89e17671a51b None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:58.667944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0b336-7e06-46ef-9889-89e17671a51b None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:58.668260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0b336-7e06-46ef-9889-89e17671a51b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:58.669411 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c6a09-d9e9-4488-a580-a9017fd8f30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:58.669674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0b336-7e06-46ef-9889-89e17671a51b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:58.669779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c6a09-d9e9-4488-a580-a9017fd8f30e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:58.670007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c6a09-d9e9-4488-a580-a9017fd8f30e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:58.670313 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c6a09-d9e9-4488-a580-a9017fd8f30e None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:58.670955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c6a09-d9e9-4488-a580-a9017fd8f30e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:58.671460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c6a09-d9e9-4488-a580-a9017fd8f30e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:58.673764 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c6a09-d9e9-4488-a580-a9017fd8f30e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:58.680233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 420/834] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:11:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:58.680754 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 415/835] 10.222.0.55 () {56 vars in 1124 bytes} [Tue Sep 27 17:11:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:58.688369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0b336-7e06-46ef-9889-89e17671a51b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:58.689805 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c6a09-d9e9-4488-a580-a9017fd8f30e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:58.720406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:58.809602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c6a09-d9e9-4488-a580-a9017fd8f30e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:58.816094 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0b336-7e06-46ef-9889-89e17671a51b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:58.853450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755c7ce-2dda-4926-bf41-7ee358e1762d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:58.854474 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755c7ce-2dda-4926-bf41-7ee358e1762d None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:58.854962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755c7ce-2dda-4926-bf41-7ee358e1762d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:58.855682 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755c7ce-2dda-4926-bf41-7ee358e1762d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:58.856589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755c7ce-2dda-4926-bf41-7ee358e1762d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:58.856908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968eae27-dd88-4783-921f-953783cd44c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:58.857562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eae27-dd88-4783-921f-953783cd44c0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:58.857839 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eae27-dd88-4783-921f-953783cd44c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:58.858132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eae27-dd88-4783-921f-953783cd44c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:58.859121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755c7ce-2dda-4926-bf41-7ee358e1762d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:58.873676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968eae27-dd88-4783-921f-953783cd44c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:58.876166 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968eae27-dd88-4783-921f-953783cd44c0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:58.889737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 421/836] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:11:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-968eae27-dd88-4783-921f-953783cd44c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:58.898289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:58.899724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:58.900679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:58.900679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:58.900679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:58.900679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 416/837] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:58.902535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755c7ce-2dda-4926-bf41-7ee358e1762d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:58.941741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:58.968439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59358fbb-b21a-478a-9e49-e10db8e16355 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:58.968771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59358fbb-b21a-478a-9e49-e10db8e16355 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:58.969035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59358fbb-b21a-478a-9e49-e10db8e16355 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:58.970762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 417/838] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:11:58.976484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fefa74-9e39-40f5-b6ae-0fe4966dd20d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:58.976762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fefa74-9e39-40f5-b6ae-0fe4966dd20d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:58.977039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fefa74-9e39-40f5-b6ae-0fe4966dd20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:58.998221 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-97fefa74-9e39-40f5-b6ae-0fe4966dd20d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:11:58.999752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-97fefa74-9e39-40f5-b6ae-0fe4966dd20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:11:59.033209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755c7ce-2dda-4926-bf41-7ee358e1762d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:59.082897 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10cf17-b470-4bea-a8de-75aa8f7c193c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:59.083520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10cf17-b470-4bea-a8de-75aa8f7c193c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:59.083797 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10cf17-b470-4bea-a8de-75aa8f7c193c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:59.084150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10cf17-b470-4bea-a8de-75aa8f7c193c None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:59.084713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10cf17-b470-4bea-a8de-75aa8f7c193c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:59.085014 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10cf17-b470-4bea-a8de-75aa8f7c193c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:59.086721 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10cf17-b470-4bea-a8de-75aa8f7c193c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:59.093500 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 418/839] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:11:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:11:59.100930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 422/840] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:59.106382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fefa74-9e39-40f5-b6ae-0fe4966dd20d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:59.110103 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:59.209473 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fefa74-9e39-40f5-b6ae-0fe4966dd20d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:59.244533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd3818-1a9e-495f-874b-a5bc3916b10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:59.245512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd3818-1a9e-495f-874b-a5bc3916b10d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:59.245990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd3818-1a9e-495f-874b-a5bc3916b10d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:59.246489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd3818-1a9e-495f-874b-a5bc3916b10d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:59.258289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd3818-1a9e-495f-874b-a5bc3916b10d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:59.259739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd3818-1a9e-495f-874b-a5bc3916b10d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-eddd3818-1a9e-495f-874b-a5bc3916b10d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:11:59.273140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:59.274126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:59.275165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:59.275165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:59.275165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:11:59.275165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:11:59.275165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:11:59.275165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:11:59.275165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:11:59.275165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:11:59.275165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 17:11:59.275165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:59.275165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 419/841] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Sep 27 17:11:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:11:59.279915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10cf17-b470-4bea-a8de-75aa8f7c193c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:59.379787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd3818-1a9e-495f-874b-a5bc3916b10d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:59.380019 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:59.392418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10cf17-b470-4bea-a8de-75aa8f7c193c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:59.428422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb40131-1185-41e8-9ecf-d408b013f4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:59.429369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb40131-1185-41e8-9ecf-d408b013f4fd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:59.429822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb40131-1185-41e8-9ecf-d408b013f4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:59.430371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb40131-1185-41e8-9ecf-d408b013f4fd None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:59.431328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb40131-1185-41e8-9ecf-d408b013f4fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:59.431822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb40131-1185-41e8-9ecf-d408b013f4fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:59.433408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb40131-1185-41e8-9ecf-d408b013f4fd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:59.449241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:59.451614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 423/842] 10.222.0.55 () {56 vars in 1149 bytes} [Tue Sep 27 17:11:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:59.457456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb40131-1185-41e8-9ecf-d408b013f4fd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:59.492034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd3818-1a9e-495f-874b-a5bc3916b10d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:59.530714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339a21-2456-4b27-9de5-6406443afa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:59.532156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339a21-2456-4b27-9de5-6406443afa4a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:59.532645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339a21-2456-4b27-9de5-6406443afa4a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:59.533091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339a21-2456-4b27-9de5-6406443afa4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:59.544513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4339a21-2456-4b27-9de5-6406443afa4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-e4339a21-2456-4b27-9de5-6406443afa4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:11:59.550039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:59.550998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:59.551870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:59.551870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:59.551870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:59.551870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:59.551870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:11:59.551870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:11:59.551870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:11:59.551870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:11:59.551870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:11:59.551870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:11:59.551870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:11:59.551870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:59.559479 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 420/843] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:11:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:59.560612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339a21-2456-4b27-9de5-6406443afa4a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:59.566137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb40131-1185-41e8-9ecf-d408b013f4fd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:59.602729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59fd14-72e1-46f8-b3d7-cde174f65854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:59.603524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59fd14-72e1-46f8-b3d7-cde174f65854 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:59.603866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59fd14-72e1-46f8-b3d7-cde174f65854 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:59.604225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59fd14-72e1-46f8-b3d7-cde174f65854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:59.622283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:59.676618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339a21-2456-4b27-9de5-6406443afa4a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:59.685461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59fd14-72e1-46f8-b3d7-cde174f65854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:59.688235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59fd14-72e1-46f8-b3d7-cde174f65854 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:59.701522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:59.715205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b838e30-55ef-430c-9fa4-1f3282b51353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:59.715854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b838e30-55ef-430c-9fa4-1f3282b51353 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:59.716217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b838e30-55ef-430c-9fa4-1f3282b51353 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:59.716490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b838e30-55ef-430c-9fa4-1f3282b51353 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:59.717266 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b838e30-55ef-430c-9fa4-1f3282b51353 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:11:59.717634 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b838e30-55ef-430c-9fa4-1f3282b51353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:59.719899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b838e30-55ef-430c-9fa4-1f3282b51353 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:59.733493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 421/844] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:11:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:11:59.734848 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b838e30-55ef-430c-9fa4-1f3282b51353 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:59.812922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 424/845] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:11:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:11:59.820691 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59fd14-72e1-46f8-b3d7-cde174f65854 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:59.840594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b838e30-55ef-430c-9fa4-1f3282b51353 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:59.874680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:59.877745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4390d17-1275-4a07-8c89-78be297ef091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:59.878841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4390d17-1275-4a07-8c89-78be297ef091 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:59.879311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4390d17-1275-4a07-8c89-78be297ef091 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:59.879764 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4390d17-1275-4a07-8c89-78be297ef091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:11:59.893586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4390d17-1275-4a07-8c89-78be297ef091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:11:59.895316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4390d17-1275-4a07-8c89-78be297ef091 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-e4390d17-1275-4a07-8c89-78be297ef091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:11:59.910580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:11:59.912142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:11:59.913042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:11:59.913042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:11:59.913042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:11:59.915113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 422/846] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:11:59.926111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4390d17-1275-4a07-8c89-78be297ef091 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:11:59.941054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59fd14-72e1-46f8-b3d7-cde174f65854 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:11:59.966429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:11:59.992712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fbedb-2578-40f6-9865-36e22b80acca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:11:59.994116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fbedb-2578-40f6-9865-36e22b80acca None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:11:59.994619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fbedb-2578-40f6-9865-36e22b80acca None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:11:59.994901 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160fbedb-2578-40f6-9865-36e22b80acca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:00.008130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160fbedb-2578-40f6-9865-36e22b80acca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:00.009836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160fbedb-2578-40f6-9865-36e22b80acca None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-160fbedb-2578-40f6-9865-36e22b80acca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:00.025779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:00.026786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:00.027679 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:00.027679 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:00.027679 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:00.029743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 425/847] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:11:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:00.038105 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fbedb-2578-40f6-9865-36e22b80acca None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:00.043288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4390d17-1275-4a07-8c89-78be297ef091 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:00.093362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8767b2-ec72-4a6e-b468-6149bc0acc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:00.094344 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8767b2-ec72-4a6e-b468-6149bc0acc1d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:00.094624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8767b2-ec72-4a6e-b468-6149bc0acc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:00.095273 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8767b2-ec72-4a6e-b468-6149bc0acc1d None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:00.095925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8767b2-ec72-4a6e-b468-6149bc0acc1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:00.096227 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8767b2-ec72-4a6e-b468-6149bc0acc1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:00.098088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8767b2-ec72-4a6e-b468-6149bc0acc1d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:00.114439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 423/848] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:00.122979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8767b2-ec72-4a6e-b468-6149bc0acc1d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:00.149502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:00.153911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160fbedb-2578-40f6-9865-36e22b80acca None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:00.192904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb5170-574b-4bad-9f08-7d01b61aaa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:00.193507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb5170-574b-4bad-9f08-7d01b61aaa81 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:00.193780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb5170-574b-4bad-9f08-7d01b61aaa81 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:00.194048 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb5170-574b-4bad-9f08-7d01b61aaa81 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:00.194716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb5170-574b-4bad-9f08-7d01b61aaa81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:00.195015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb5170-574b-4bad-9f08-7d01b61aaa81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:00.196364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb5170-574b-4bad-9f08-7d01b61aaa81 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:00.216158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 426/849] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:00.217478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb5170-574b-4bad-9f08-7d01b61aaa81 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:00.246388 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:00.251707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8767b2-ec72-4a6e-b468-6149bc0acc1d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:00.289328 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee4059-258a-48d7-93e6-efc810aa517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:00.290437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee4059-258a-48d7-93e6-efc810aa517c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:00.290817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee4059-258a-48d7-93e6-efc810aa517c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:00.291178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee4059-258a-48d7-93e6-efc810aa517c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:00.303252 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee4059-258a-48d7-93e6-efc810aa517c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:00.305011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee4059-258a-48d7-93e6-efc810aa517c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-f1ee4059-258a-48d7-93e6-efc810aa517c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:12:00.317950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:00.319244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:00.320319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:00.320319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:00.320319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:12:00.320319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:12:00.320319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:12:00.320319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:12:00.320319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:12:00.320319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:12:00.320319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 27 17:12:00.320319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:00.322488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 424/850] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:12:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:00.331072 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee4059-258a-48d7-93e6-efc810aa517c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:00.344451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb5170-574b-4bad-9f08-7d01b61aaa81 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:00.387948 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0033275-8cde-44cd-9431-640f202872e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:00.388724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0033275-8cde-44cd-9431-640f202872e0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:00.389066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0033275-8cde-44cd-9431-640f202872e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:00.389400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0033275-8cde-44cd-9431-640f202872e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:00.396884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0033275-8cde-44cd-9431-640f202872e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:00.399125 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0033275-8cde-44cd-9431-640f202872e0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d0033275-8cde-44cd-9431-640f202872e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:12:00.411838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:00.413020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:00.414175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:12:00.414175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:12:00.414175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:12:00.414175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:12:00.414175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:12:00.414175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:12:00.414175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:00.414175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:00.415276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 427/851] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:12:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:00.420508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0033275-8cde-44cd-9431-640f202872e0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:00.442196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:00.468686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee4059-258a-48d7-93e6-efc810aa517c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:00.514229 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730bbd2-d912-4514-a6a2-bff1658e318e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:00.514950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730bbd2-d912-4514-a6a2-bff1658e318e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:00.515300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730bbd2-d912-4514-a6a2-bff1658e318e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:00.515675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730bbd2-d912-4514-a6a2-bff1658e318e None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:00.516483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730bbd2-d912-4514-a6a2-bff1658e318e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:00.516860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730bbd2-d912-4514-a6a2-bff1658e318e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:00.519102 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730bbd2-d912-4514-a6a2-bff1658e318e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:00.529532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0033275-8cde-44cd-9431-640f202872e0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:00.545345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 425/852] 10.222.0.55 () {56 vars in 1135 bytes} [Tue Sep 27 17:12:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:00.547771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730bbd2-d912-4514-a6a2-bff1658e318e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:00.567558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:00.571527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823b29f-6adf-4301-9f88-e4aa306196a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:00.572135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823b29f-6adf-4301-9f88-e4aa306196a7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:00.572642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823b29f-6adf-4301-9f88-e4aa306196a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:00.572937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823b29f-6adf-4301-9f88-e4aa306196a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:00.574443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823b29f-6adf-4301-9f88-e4aa306196a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:00.575219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823b29f-6adf-4301-9f88-e4aa306196a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:00.576936 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823b29f-6adf-4301-9f88-e4aa306196a7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:00.598239 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 428/853] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:12:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:00.599985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823b29f-6adf-4301-9f88-e4aa306196a7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:00.665313 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730bbd2-d912-4514-a6a2-bff1658e318e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:00.705363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cae8a-e17e-403b-ae6c-8b8c0820e33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:00.705973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cae8a-e17e-403b-ae6c-8b8c0820e33f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:00.706327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cae8a-e17e-403b-ae6c-8b8c0820e33f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:00.706604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772cae8a-e17e-403b-ae6c-8b8c0820e33f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:00.728865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:00.761409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823b29f-6adf-4301-9f88-e4aa306196a7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:00.789918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cae8a-e17e-403b-ae6c-8b8c0820e33f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:00.791703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772cae8a-e17e-403b-ae6c-8b8c0820e33f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:00.806540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64515f29-1775-4044-bb02-b8881d3bea25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:00.807277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64515f29-1775-4044-bb02-b8881d3bea25 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:00.807644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64515f29-1775-4044-bb02-b8881d3bea25 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:00.808272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64515f29-1775-4044-bb02-b8881d3bea25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:00.809110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64515f29-1775-4044-bb02-b8881d3bea25 None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=c8c3fcd3789c40c88a4c32fff2678fc9) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:00.809951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64515f29-1775-4044-bb02-b8881d3bea25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:00.811740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64515f29-1775-4044-bb02-b8881d3bea25 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:00.812564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:00.822182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 429/854] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:12:00 2022] GET /identity/v3/role_assignments?user.id=c8c3fcd3789c40c88a4c32fff2678fc9&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:00.842998 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ea725-8036-4d7c-9280-4fc641fb338d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:00.844053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ea725-8036-4d7c-9280-4fc641fb338d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:00.844559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ea725-8036-4d7c-9280-4fc641fb338d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:00.846427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 430/855] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:00.852369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff372eef-6785-42f1-81cc-1d775c01ca56 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:00.852549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff372eef-6785-42f1-81cc-1d775c01ca56 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:00.852821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff372eef-6785-42f1-81cc-1d775c01ca56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:00.873671 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ff372eef-6785-42f1-81cc-1d775c01ca56 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:00.875451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ff372eef-6785-42f1-81cc-1d775c01ca56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:00.927338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 426/856] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:12:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:12:00.930167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cae8a-e17e-403b-ae6c-8b8c0820e33f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:00.962859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 431/857] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:01.051038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772cae8a-e17e-403b-ae6c-8b8c0820e33f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:01.068389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:01.088520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86c4b1-ae6f-4490-93b7-715347048992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:01.089449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86c4b1-ae6f-4490-93b7-715347048992 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:01.089925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86c4b1-ae6f-4490-93b7-715347048992 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:01.090427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86c4b1-ae6f-4490-93b7-715347048992 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:01.101233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86c4b1-ae6f-4490-93b7-715347048992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:01.103158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86c4b1-ae6f-4490-93b7-715347048992 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-dd86c4b1-ae6f-4490-93b7-715347048992 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:01.120084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:01.121083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:01.121998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:01.121998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:01.121998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:01.122474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 427/858] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:01.129979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff372eef-6785-42f1-81cc-1d775c01ca56 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:01.208231 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:01.238353 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff372eef-6785-42f1-81cc-1d775c01ca56 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:01.273266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610aef1-3115-4f34-9faa-32aaf06413ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:01.273914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610aef1-3115-4f34-9faa-32aaf06413ae None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:01.274215 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610aef1-3115-4f34-9faa-32aaf06413ae None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:01.274482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610aef1-3115-4f34-9faa-32aaf06413ae None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:01.275175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610aef1-3115-4f34-9faa-32aaf06413ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:01.275519 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610aef1-3115-4f34-9faa-32aaf06413ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:01.277192 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610aef1-3115-4f34-9faa-32aaf06413ae None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:01.277800 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86c4b1-ae6f-4490-93b7-715347048992 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:01.293194 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 432/859] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:01.299414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610aef1-3115-4f34-9faa-32aaf06413ae None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:01.391393 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86c4b1-ae6f-4490-93b7-715347048992 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:01.407738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610aef1-3115-4f34-9faa-32aaf06413ae None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:01.426338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb14b2-388d-477f-b482-275290a9aa8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:01.426966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb14b2-388d-477f-b482-275290a9aa8e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:01.427342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb14b2-388d-477f-b482-275290a9aa8e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:01.427509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb14b2-388d-477f-b482-275290a9aa8e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:01.428077 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb14b2-388d-477f-b482-275290a9aa8e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:01.429450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb14b2-388d-477f-b482-275290a9aa8e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-8feb14b2-388d-477f-b482-275290a9aa8e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:01.438401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:01.439353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:01.440367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 428/860] 10.222.0.55 () {56 vars in 1132 bytes} [Tue Sep 27 17:12:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:01.441188 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a528594-158c-4b3d-848c-0c2eb2a18fec None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:01.441188 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a528594-158c-4b3d-848c-0c2eb2a18fec None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:01.441188 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a528594-158c-4b3d-848c-0c2eb2a18fec None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:01.446707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 429/861] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:01.446707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a528594-158c-4b3d-848c-0c2eb2a18fec None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:01.447716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:01.453345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7b2e4-fe6e-4a33-9ba2-4700ce72d45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:01.453968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7b2e4-fe6e-4a33-9ba2-4700ce72d45e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:01.454652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7b2e4-fe6e-4a33-9ba2-4700ce72d45e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:01.455418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7b2e4-fe6e-4a33-9ba2-4700ce72d45e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:01.463735 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7b2e4-fe6e-4a33-9ba2-4700ce72d45e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:01.465263 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7b2e4-fe6e-4a33-9ba2-4700ce72d45e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-49f7b2e4-fe6e-4a33-9ba2-4700ce72d45e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:12:01.477218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:01.478126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:01.479031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:12:01.479031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:12:01.479031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:12:01.479031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:12:01.479031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:12:01.479031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:12:01.479031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:01.479031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:01.479031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 433/862] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:12:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:01.480484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f37767-3174-4a1e-8d47-4666a5aca7de None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:01.480484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f37767-3174-4a1e-8d47-4666a5aca7de None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:01.480654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f37767-3174-4a1e-8d47-4666a5aca7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:01.481117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:01.498484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-92f37767-3174-4a1e-8d47-4666a5aca7de None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:01.500208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-92f37767-3174-4a1e-8d47-4666a5aca7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:01.558338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a528594-158c-4b3d-848c-0c2eb2a18fec None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:01.570743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 434/863] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:01.571706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f37767-3174-4a1e-8d47-4666a5aca7de None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:01.591897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091775b7-bfaf-4472-90e7-b7d434b0429b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:01.592718 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091775b7-bfaf-4472-90e7-b7d434b0429b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:01.593193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091775b7-bfaf-4472-90e7-b7d434b0429b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:01.594160 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091775b7-bfaf-4472-90e7-b7d434b0429b None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:01.594921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091775b7-bfaf-4472-90e7-b7d434b0429b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:01.595356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091775b7-bfaf-4472-90e7-b7d434b0429b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:01.596850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091775b7-bfaf-4472-90e7-b7d434b0429b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:01.601406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 430/864] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Sep 27 17:12:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:01.607151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091775b7-bfaf-4472-90e7-b7d434b0429b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:01.706888 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f37767-3174-4a1e-8d47-4666a5aca7de None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:01.718338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:01.726710 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091775b7-bfaf-4472-90e7-b7d434b0429b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:01.739560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:01.757680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf9edd-034c-4106-b9e4-006a21ec3101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:01.759423 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf9edd-034c-4106-b9e4-006a21ec3101 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:01.759861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf9edd-034c-4106-b9e4-006a21ec3101 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:01.760588 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf9edd-034c-4106-b9e4-006a21ec3101 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:01.761685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf9edd-034c-4106-b9e4-006a21ec3101 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:01.762185 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf9edd-034c-4106-b9e4-006a21ec3101 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:01.764699 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf9edd-034c-4106-b9e4-006a21ec3101 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:01.768613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d57b1-21aa-4201-9b51-a6fb3cab8bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:01.769714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d57b1-21aa-4201-9b51-a6fb3cab8bcc None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:01.770321 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d57b1-21aa-4201-9b51-a6fb3cab8bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:01.770875 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d57b1-21aa-4201-9b51-a6fb3cab8bcc None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:01.771988 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d57b1-21aa-4201-9b51-a6fb3cab8bcc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:01.772575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d57b1-21aa-4201-9b51-a6fb3cab8bcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:01.774741 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d57b1-21aa-4201-9b51-a6fb3cab8bcc None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:01.786198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 431/865] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Sep 27 17:12:01 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:01.786351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 435/866] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:12:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:01.791129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d57b1-21aa-4201-9b51-a6fb3cab8bcc None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:01.853287 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf9edd-034c-4106-b9e4-006a21ec3101 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:01.911909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d57b1-21aa-4201-9b51-a6fb3cab8bcc None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:01.950242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e89ec-f6f4-45a7-8198-cb2627d03ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:01.951738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e89ec-f6f4-45a7-8198-cb2627d03ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:01.952230 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e89ec-f6f4-45a7-8198-cb2627d03ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:01.952701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e89ec-f6f4-45a7-8198-cb2627d03ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:01.954375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e89ec-f6f4-45a7-8198-cb2627d03ed4 None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=39073006a0054b56bb1bba46d2d0768e) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:01.954944 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e89ec-f6f4-45a7-8198-cb2627d03ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:01.956651 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e89ec-f6f4-45a7-8198-cb2627d03ed4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:01.963712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 432/867] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:12:01 2022] GET /identity/v3/role_assignments?user.id=39073006a0054b56bb1bba46d2d0768e&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:01.988958 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf9edd-034c-4106-b9e4-006a21ec3101 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:02.030472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f578c-3700-41e4-8457-14da1643ea93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:02.031649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f578c-3700-41e4-8457-14da1643ea93 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:02.031930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f578c-3700-41e4-8457-14da1643ea93 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:02.032200 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f578c-3700-41e4-8457-14da1643ea93 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:02.032819 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f578c-3700-41e4-8457-14da1643ea93 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:02.034214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f578c-3700-41e4-8457-14da1643ea93 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-a18f578c-3700-41e4-8457-14da1643ea93 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:02.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:02.045233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:02.046108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 436/868] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:12:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:02.051708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e89ec-f6f4-45a7-8198-cb2627d03ed4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:02.083253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:02.169131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e89ec-f6f4-45a7-8198-cb2627d03ed4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:02.205403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1051cbd-9948-42f3-86bf-76863f2370e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:02.206022 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1051cbd-9948-42f3-86bf-76863f2370e1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:02.206357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1051cbd-9948-42f3-86bf-76863f2370e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:02.206632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1051cbd-9948-42f3-86bf-76863f2370e1 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:02.207272 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1051cbd-9948-42f3-86bf-76863f2370e1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:02.207574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1051cbd-9948-42f3-86bf-76863f2370e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:02.208941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1051cbd-9948-42f3-86bf-76863f2370e1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:02.214566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 433/869] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:12:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:02.220810 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f578c-3700-41e4-8457-14da1643ea93 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:02.270478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:02.325208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f578c-3700-41e4-8457-14da1643ea93 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:02.360556 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb1bcd-12f5-4463-b760-38c0dd7a62c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:02.361156 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1bcd-12f5-4463-b760-38c0dd7a62c4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:02.361428 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1bcd-12f5-4463-b760-38c0dd7a62c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:02.361688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb1bcd-12f5-4463-b760-38c0dd7a62c4 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:02.362339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb1bcd-12f5-4463-b760-38c0dd7a62c4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:02.362632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb1bcd-12f5-4463-b760-38c0dd7a62c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:02.363979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb1bcd-12f5-4463-b760-38c0dd7a62c4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:02.368939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 437/870] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:12:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:03.185436 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53352cf9-4a47-4a76-a8b9-68dd2267c145 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:03.186322 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53352cf9-4a47-4a76-a8b9-68dd2267c145 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:03.186767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53352cf9-4a47-4a76-a8b9-68dd2267c145 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:03.189063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 434/871] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:03.194820 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ae39a-d0c0-431b-84b8-218398a0de25 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:03.195550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ae39a-d0c0-431b-84b8-218398a0de25 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:03.195969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ae39a-d0c0-431b-84b8-218398a0de25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:03.215033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-dd2ae39a-d0c0-431b-84b8-218398a0de25 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:03.216416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dd2ae39a-d0c0-431b-84b8-218398a0de25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:03.288298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 438/872] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:03.558207 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53352cf9-4a47-4a76-a8b9-68dd2267c145 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:03.651893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:03.668074 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53352cf9-4a47-4a76-a8b9-68dd2267c145 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:03.703834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5483830-0662-4a86-9d58-c4d7b1689e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:03.704926 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5483830-0662-4a86-9d58-c4d7b1689e82 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:03.705407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5483830-0662-4a86-9d58-c4d7b1689e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:03.705885 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5483830-0662-4a86-9d58-c4d7b1689e82 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:03.716862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5483830-0662-4a86-9d58-c4d7b1689e82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a5483830-0662-4a86-9d58-c4d7b1689e82 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:12:03.722476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:03.723396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:03.724253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:03.724253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:03.724253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:03.724253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:03.724253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:12:03.724253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:12:03.724253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:12:03.724253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:12:03.724253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:12:03.724253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:12:03.724253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 17:12:03.724253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:03.731720 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 435/873] 10.222.0.55 () {56 vars in 1120 bytes} [Tue Sep 27 17:12:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:03.733311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ae39a-d0c0-431b-84b8-218398a0de25 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:03.777823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:03.837745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ae39a-d0c0-431b-84b8-218398a0de25 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:03.872475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92371db7-5207-4351-866e-cea324fac812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:03.873077 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92371db7-5207-4351-866e-cea324fac812 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:03.873350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92371db7-5207-4351-866e-cea324fac812 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:03.873615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92371db7-5207-4351-866e-cea324fac812 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:03.874305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92371db7-5207-4351-866e-cea324fac812 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:03.875179 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92371db7-5207-4351-866e-cea324fac812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:03.876536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92371db7-5207-4351-866e-cea324fac812 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:03.884939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 439/874] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Sep 27 17:12:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:03.890788 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5483830-0662-4a86-9d58-c4d7b1689e82 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:04.001873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5483830-0662-4a86-9d58-c4d7b1689e82 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:04.037963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e04fc7-3df6-4758-b8d9-7c6284ce36ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:04.039027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e04fc7-3df6-4758-b8d9-7c6284ce36ce None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:04.039414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e04fc7-3df6-4758-b8d9-7c6284ce36ce None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:04.039689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e04fc7-3df6-4758-b8d9-7c6284ce36ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:04.052374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e04fc7-3df6-4758-b8d9-7c6284ce36ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:04.053836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e04fc7-3df6-4758-b8d9-7c6284ce36ce None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:04.062609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-b6e04fc7-3df6-4758-b8d9-7c6284ce36ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:04.069516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:04.070685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:04.072081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:04.072081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:04.072081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:04.072081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 436/875] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:04.078989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92371db7-5207-4351-866e-cea324fac812 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:04.104651 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06dd76-0cf4-4b8d-a388-a29be9622ada None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:04.104988 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06dd76-0cf4-4b8d-a388-a29be9622ada None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:04.105454 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06dd76-0cf4-4b8d-a388-a29be9622ada None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:04.107139 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 437/876] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:04.114338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76fc0c1-41ff-4844-bc20-38415ce40d48 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:04.114842 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76fc0c1-41ff-4844-bc20-38415ce40d48 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:04.115330 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76fc0c1-41ff-4844-bc20-38415ce40d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:04.135241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a76fc0c1-41ff-4844-bc20-38415ce40d48 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:04.136519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a76fc0c1-41ff-4844-bc20-38415ce40d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:04.201609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92371db7-5207-4351-866e-cea324fac812 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:04.203147 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:04.234414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 438/877] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:04.242689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f698d-089a-41fe-8887-4f0536fda48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:04.243300 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f698d-089a-41fe-8887-4f0536fda48d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:04.243579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f698d-089a-41fe-8887-4f0536fda48d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:04.243850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f698d-089a-41fe-8887-4f0536fda48d None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:04.244498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f698d-089a-41fe-8887-4f0536fda48d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:04.244795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f698d-089a-41fe-8887-4f0536fda48d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:04.246177 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766f698d-089a-41fe-8887-4f0536fda48d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:04.251212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2958617-c80a-43f0-a4d2-16431c7078bc None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:04.251499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2958617-c80a-43f0-a4d2-16431c7078bc None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:04.251775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2958617-c80a-43f0-a4d2-16431c7078bc None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:04.253348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 439/878] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:04.258400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aebbe-909a-4deb-a1fd-ac11e5b294ac None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:04.258683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aebbe-909a-4deb-a1fd-ac11e5b294ac None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:04.258958 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aebbe-909a-4deb-a1fd-ac11e5b294ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:04.262166 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 440/879] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:04.266375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f698d-089a-41fe-8887-4f0536fda48d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:04.281494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-177aebbe-909a-4deb-a1fd-ac11e5b294ac None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:04.282730 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-177aebbe-909a-4deb-a1fd-ac11e5b294ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:04.371381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 440/880] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:04.394794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766f698d-089a-41fe-8887-4f0536fda48d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:04.430554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350d964-bb6f-46e3-8b5a-30f1f9394ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:04.431175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350d964-bb6f-46e3-8b5a-30f1f9394ebd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:04.431451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350d964-bb6f-46e3-8b5a-30f1f9394ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:04.431719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350d964-bb6f-46e3-8b5a-30f1f9394ebd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:04.439754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350d964-bb6f-46e3-8b5a-30f1f9394ebd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:04.441184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350d964-bb6f-46e3-8b5a-30f1f9394ebd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-5350d964-bb6f-46e3-8b5a-30f1f9394ebd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:12:04.453504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:04.455580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:04.457593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:04.457593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:04.457593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:12:04.457593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:12:04.457593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:12:04.457593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:12:04.457593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:12:04.457593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:12:04.457593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 17:12:04.457593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:04.457593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 441/881] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Sep 27 17:12:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:04.463329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aebbe-909a-4deb-a1fd-ac11e5b294ac None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:04.509888 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350d964-bb6f-46e3-8b5a-30f1f9394ebd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:04.525739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:04.540516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:04.594573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aebbe-909a-4deb-a1fd-ac11e5b294ac None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:04.630850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc3a6b-85a9-410e-b117-df53545144d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:04.631179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc3a6b-85a9-410e-b117-df53545144d2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:04.631389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc3a6b-85a9-410e-b117-df53545144d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:04.631654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc3a6b-85a9-410e-b117-df53545144d2 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:04.632401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc3a6b-85a9-410e-b117-df53545144d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:04.632699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc3a6b-85a9-410e-b117-df53545144d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:04.634095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc3a6b-85a9-410e-b117-df53545144d2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:04.642745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350d964-bb6f-46e3-8b5a-30f1f9394ebd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:04.652140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 441/882] 10.222.0.55 () {56 vars in 1149 bytes} [Tue Sep 27 17:12:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:04.657458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc3a6b-85a9-410e-b117-df53545144d2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:04.684329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8b184-617a-473e-a2e7-a159bf141947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:04.684934 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8b184-617a-473e-a2e7-a159bf141947 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:04.685213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8b184-617a-473e-a2e7-a159bf141947 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:04.685484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8b184-617a-473e-a2e7-a159bf141947 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:04.686157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f8b184-617a-473e-a2e7-a159bf141947 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:04.687588 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f8b184-617a-473e-a2e7-a159bf141947 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:04.702045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 442/883] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:12:04 2022] POST /identity/v3/services => generated 269 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:12:04.703793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a78cb1-de0b-42a4-a9aa-b7e1f88f0785 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:04.704081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a78cb1-de0b-42a4-a9aa-b7e1f88f0785 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:04.704331 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a78cb1-de0b-42a4-a9aa-b7e1f88f0785 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:04.710030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 443/884] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:04.710030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a78cb1-de0b-42a4-a9aa-b7e1f88f0785 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:04.773381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc3a6b-85a9-410e-b117-df53545144d2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:04.793400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:04.809065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:04.819866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5589a2-7461-43ff-8719-7f0a23e818df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:04.820483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5589a2-7461-43ff-8719-7f0a23e818df None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:04.820765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5589a2-7461-43ff-8719-7f0a23e818df None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:04.821037 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5589a2-7461-43ff-8719-7f0a23e818df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:04.835434 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a78cb1-de0b-42a4-a9aa-b7e1f88f0785 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:04.874429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5641c-c052-4a24-9f7c-fa94c96a08b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:04.875273 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5641c-c052-4a24-9f7c-fa94c96a08b0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:04.875551 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5641c-c052-4a24-9f7c-fa94c96a08b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:04.875823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5641c-c052-4a24-9f7c-fa94c96a08b0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:04.876412 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5641c-c052-4a24-9f7c-fa94c96a08b0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:04.877768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5641c-c052-4a24-9f7c-fa94c96a08b0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-76f5641c-c052-4a24-9f7c-fa94c96a08b0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:04.888150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:04.889197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:04.890703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 444/885] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Sep 27 17:12:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:04.892104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef49d0-dd69-4cb6-93c9-032e9c7c3844 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:04.892104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef49d0-dd69-4cb6-93c9-032e9c7c3844 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:04.892104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef49d0-dd69-4cb6-93c9-032e9c7c3844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:04.898282 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5589a2-7461-43ff-8719-7f0a23e818df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:04.899748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5589a2-7461-43ff-8719-7f0a23e818df None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:04.913816 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-55ef49d0-dd69-4cb6-93c9-032e9c7c3844 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:04.915193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-55ef49d0-dd69-4cb6-93c9-032e9c7c3844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:05.007325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 442/886] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:12:04 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:12:05.018094 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 445/887] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:04 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:05.018275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5589a2-7461-43ff-8719-7f0a23e818df None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:05.018365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef49d0-dd69-4cb6-93c9-032e9c7c3844 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:05.029852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:05.113410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:05.134723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5589a2-7461-43ff-8719-7f0a23e818df None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:05.142381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef49d0-dd69-4cb6-93c9-032e9c7c3844 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:05.173843 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb5276-9d42-4910-9aaa-4c8c4f1c158a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:05.174569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb5276-9d42-4910-9aaa-4c8c4f1c158a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:05.174914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb5276-9d42-4910-9aaa-4c8c4f1c158a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:05.175254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb5276-9d42-4910-9aaa-4c8c4f1c158a None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:05.176139 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb5276-9d42-4910-9aaa-4c8c4f1c158a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:05.176576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb5276-9d42-4910-9aaa-4c8c4f1c158a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:05.178025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb5276-9d42-4910-9aaa-4c8c4f1c158a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:05.182264 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f15b6-5e6e-42f2-aead-6827462af770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:05.182899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f15b6-5e6e-42f2-aead-6827462af770 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:05.183177 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f15b6-5e6e-42f2-aead-6827462af770 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:05.183387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 443/888] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Sep 27 17:12:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:05.183535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f15b6-5e6e-42f2-aead-6827462af770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:05.191057 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb5276-9d42-4910-9aaa-4c8c4f1c158a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:05.196406 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f15b6-5e6e-42f2-aead-6827462af770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:05.197859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f15b6-5e6e-42f2-aead-6827462af770 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-488f15b6-5e6e-42f2-aead-6827462af770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:05.213778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:05.215426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:05.216548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:05.216548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:05.216548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:05.220178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 446/889] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:05.221461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f15b6-5e6e-42f2-aead-6827462af770 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:05.280254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:05.309609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb5276-9d42-4910-9aaa-4c8c4f1c158a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:05.347650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f59f3-8e98-4681-bd36-b33c6d5beec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:05.348247 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f59f3-8e98-4681-bd36-b33c6d5beec3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:05.348520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f59f3-8e98-4681-bd36-b33c6d5beec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:05.348784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f59f3-8e98-4681-bd36-b33c6d5beec3 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:05.349402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f59f3-8e98-4681-bd36-b33c6d5beec3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:05.349699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f59f3-8e98-4681-bd36-b33c6d5beec3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:05.349888 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f15b6-5e6e-42f2-aead-6827462af770 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:05.351086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f59f3-8e98-4681-bd36-b33c6d5beec3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:05.356517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 444/890] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Sep 27 17:12:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:05.357513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f59f3-8e98-4681-bd36-b33c6d5beec3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:05.382400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:05.389560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173dd2f-7f77-4daf-9da7-f1b7fdfbec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:05.390564 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173dd2f-7f77-4daf-9da7-f1b7fdfbec50 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:05.390853 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173dd2f-7f77-4daf-9da7-f1b7fdfbec50 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:05.391130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6173dd2f-7f77-4daf-9da7-f1b7fdfbec50 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:05.391795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173dd2f-7f77-4daf-9da7-f1b7fdfbec50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:05.392131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173dd2f-7f77-4daf-9da7-f1b7fdfbec50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:05.393792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6173dd2f-7f77-4daf-9da7-f1b7fdfbec50 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:05.409490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 447/891] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:05.415058 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173dd2f-7f77-4daf-9da7-f1b7fdfbec50 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:05.506101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f59f3-8e98-4681-bd36-b33c6d5beec3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:05.549337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6173dd2f-7f77-4daf-9da7-f1b7fdfbec50 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:05.552957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9093b-9793-47ca-b4d7-e283d6409ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:05.553557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9093b-9793-47ca-b4d7-e283d6409ed0 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:05.553832 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9093b-9793-47ca-b4d7-e283d6409ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:05.554150 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba9093b-9793-47ca-b4d7-e283d6409ed0 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:05.554810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9093b-9793-47ca-b4d7-e283d6409ed0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:05.556187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba9093b-9793-47ca-b4d7-e283d6409ed0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:05.582274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 445/892] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:12:05 2022] POST /identity/v3/services => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:05.604724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902204a-d0a2-4470-a687-e5a1df787769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:05.606031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902204a-d0a2-4470-a687-e5a1df787769 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:05.606582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902204a-d0a2-4470-a687-e5a1df787769 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:05.607134 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902204a-d0a2-4470-a687-e5a1df787769 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:05.616608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902204a-d0a2-4470-a687-e5a1df787769 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:05.618490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902204a-d0a2-4470-a687-e5a1df787769 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7902204a-d0a2-4470-a687-e5a1df787769 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:12:05.633055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:05.634089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:05.635031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:12:05.635031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:12:05.635031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:12:05.635031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:12:05.635031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:12:05.635031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:12:05.635031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:05.635031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:05.637772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 448/893] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:12:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:05.644543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9093b-9793-47ca-b4d7-e283d6409ed0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:05.647687 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:05.769976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba9093b-9793-47ca-b4d7-e283d6409ed0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:05.809380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff957a-e684-4b5d-adab-a50797a77461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:05.809987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff957a-e684-4b5d-adab-a50797a77461 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:05.810419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff957a-e684-4b5d-adab-a50797a77461 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:05.810699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff957a-e684-4b5d-adab-a50797a77461 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:05.811392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff957a-e684-4b5d-adab-a50797a77461 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:05.811699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff957a-e684-4b5d-adab-a50797a77461 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:05.813669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff957a-e684-4b5d-adab-a50797a77461 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:05.826878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 446/894] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:12:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:05.834882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902204a-d0a2-4470-a687-e5a1df787769 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:05.886828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:05.943325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902204a-d0a2-4470-a687-e5a1df787769 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:05.978096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafce72-20fd-4665-bb15-7b664aea38bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:05.978714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafce72-20fd-4665-bb15-7b664aea38bd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:05.979066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafce72-20fd-4665-bb15-7b664aea38bd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:05.979338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafce72-20fd-4665-bb15-7b664aea38bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:05.980310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafce72-20fd-4665-bb15-7b664aea38bd None admin] RBAC: Adding query filter params (role.id=a2cb336de6ca492ab46d7dbad02e31a0, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=c8c3fcd3789c40c88a4c32fff2678fc9) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:05.980621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafce72-20fd-4665-bb15-7b664aea38bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:05.982042 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafce72-20fd-4665-bb15-7b664aea38bd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:05.988098 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 449/895] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:12:05 2022] GET /identity/v3/role_assignments?user.id=c8c3fcd3789c40c88a4c32fff2678fc9&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=a2cb336de6ca492ab46d7dbad02e31a0 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:06.944415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ee6c3-b835-4b1e-af94-a327b2ee6501 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:06.944979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ee6c3-b835-4b1e-af94-a327b2ee6501 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:06.944979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ee6c3-b835-4b1e-af94-a327b2ee6501 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:06.946841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 447/896] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:06.952184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a29c14-a9eb-485c-9352-7c54d2e82f01 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:06.953248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a29c14-a9eb-485c-9352-7c54d2e82f01 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:06.953248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a29c14-a9eb-485c-9352-7c54d2e82f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:06.971333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-59a29c14-a9eb-485c-9352-7c54d2e82f01 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:06.972494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-59a29c14-a9eb-485c-9352-7c54d2e82f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:07.039672 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 450/897] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:07.306898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ee6c3-b835-4b1e-af94-a327b2ee6501 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:07.370152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:07.412582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ee6c3-b835-4b1e-af94-a327b2ee6501 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:07.448542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803e770-0194-41e6-9148-e24d489df038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:07.449174 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803e770-0194-41e6-9148-e24d489df038 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:07.449450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803e770-0194-41e6-9148-e24d489df038 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:07.449718 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803e770-0194-41e6-9148-e24d489df038 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:07.450334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803e770-0194-41e6-9148-e24d489df038 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:07.451703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803e770-0194-41e6-9148-e24d489df038 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-b803e770-0194-41e6-9148-e24d489df038 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:07.461549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:07.462569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:07.463648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 448/898] 10.222.0.55 () {56 vars in 1132 bytes} [Tue Sep 27 17:12:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:07.471277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a29c14-a9eb-485c-9352-7c54d2e82f01 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:07.528537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:07.590027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a29c14-a9eb-485c-9352-7c54d2e82f01 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:07.625309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf01caf-a755-4b56-8716-d8b3f82cb7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:07.625910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf01caf-a755-4b56-8716-d8b3f82cb7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:07.626215 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf01caf-a755-4b56-8716-d8b3f82cb7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:07.626486 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf01caf-a755-4b56-8716-d8b3f82cb7b6 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:07.627114 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf01caf-a755-4b56-8716-d8b3f82cb7b6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:07.627410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf01caf-a755-4b56-8716-d8b3f82cb7b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:07.628770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf01caf-a755-4b56-8716-d8b3f82cb7b6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:07.633822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 451/899] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Sep 27 17:12:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:07.638475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc542176-9b81-4f7a-8715-f6ff0e8f7309 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:07.639080 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf01caf-a755-4b56-8716-d8b3f82cb7b6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:07.639349 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc542176-9b81-4f7a-8715-f6ff0e8f7309 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:07.639894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc542176-9b81-4f7a-8715-f6ff0e8f7309 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:07.643383 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 449/900] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:07.647684 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a57c0e-5148-40ef-b922-0621c03a098c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:07.647971 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a57c0e-5148-40ef-b922-0621c03a098c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:07.648246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a57c0e-5148-40ef-b922-0621c03a098c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:07.670374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-12a57c0e-5148-40ef-b922-0621c03a098c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:07.672769 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-12a57c0e-5148-40ef-b922-0621c03a098c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:07.749390 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf01caf-a755-4b56-8716-d8b3f82cb7b6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:07.751767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:07.758290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 450/901] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:07.791212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020eef41-8d2a-476a-8b9e-0f7e86b50fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:07.792141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020eef41-8d2a-476a-8b9e-0f7e86b50fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:07.792632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020eef41-8d2a-476a-8b9e-0f7e86b50fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:07.793106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020eef41-8d2a-476a-8b9e-0f7e86b50fd3 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:07.793934 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020eef41-8d2a-476a-8b9e-0f7e86b50fd3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:07.794523 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020eef41-8d2a-476a-8b9e-0f7e86b50fd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:07.796066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020eef41-8d2a-476a-8b9e-0f7e86b50fd3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:07.802179 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 452/902] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Sep 27 17:12:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:07.808385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a57c0e-5148-40ef-b922-0621c03a098c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:07.883313 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:07.940426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a57c0e-5148-40ef-b922-0621c03a098c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:07.964400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320ecd1-7b33-4faa-9629-2cb6ab81f1af None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:07.964896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320ecd1-7b33-4faa-9629-2cb6ab81f1af None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:07.964996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320ecd1-7b33-4faa-9629-2cb6ab81f1af None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:07.966603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 453/903] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:07.973424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9d8f3-bcfe-43a7-91a8-880940a4d5b6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:07.973696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9d8f3-bcfe-43a7-91a8-880940a4d5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:07.973962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9d8f3-bcfe-43a7-91a8-880940a4d5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:07.982883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c1312-6eac-4e4c-ba3d-0c1fc396709e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:07.983500 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c1312-6eac-4e4c-ba3d-0c1fc396709e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:07.984287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c1312-6eac-4e4c-ba3d-0c1fc396709e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:07.984562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c1312-6eac-4e4c-ba3d-0c1fc396709e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:07.985269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c1312-6eac-4e4c-ba3d-0c1fc396709e None admin] RBAC: Adding query filter params (interface=public, service_id=ca2cb474ff5d4815b37c3040e9632d3e, region_id=RegionOne) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:07.985574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c1312-6eac-4e4c-ba3d-0c1fc396709e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:07.986959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c1312-6eac-4e4c-ba3d-0c1fc396709e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:07.992499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 451/904] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Sep 27 17:12:07 2022] GET /identity/v3/endpoints?service_id=ca2cb474ff5d4815b37c3040e9632d3e&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:07.997802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c1312-6eac-4e4c-ba3d-0c1fc396709e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:07.997996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1ba9d8f3-bcfe-43a7-91a8-880940a4d5b6 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:07.999310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1ba9d8f3-bcfe-43a7-91a8-880940a4d5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:08.072825 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 454/905] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:08.074144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9d8f3-bcfe-43a7-91a8-880940a4d5b6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:08.113427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c1312-6eac-4e4c-ba3d-0c1fc396709e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:08.139808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:08.159315 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0b7d5-c7f7-40f4-946b-3ac36a4ec3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:08.159926 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0b7d5-c7f7-40f4-946b-3ac36a4ec3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:08.160203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0b7d5-c7f7-40f4-946b-3ac36a4ec3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:08.160474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0b7d5-c7f7-40f4-946b-3ac36a4ec3a5 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:08.161954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0b7d5-c7f7-40f4-946b-3ac36a4ec3a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:08.163411 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0b7d5-c7f7-40f4-946b-3ac36a4ec3a5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:08.170936 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 452/906] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Sep 27 17:12:07 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:12:08.190972 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:08.198227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9d8f3-bcfe-43a7-91a8-880940a4d5b6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:08.203792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58066a-a540-47eb-8400-755766a9d56e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:08.204125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58066a-a540-47eb-8400-755766a9d56e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:08.204337 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58066a-a540-47eb-8400-755766a9d56e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:08.205985 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 453/907] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:08.212356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac1bce-7763-4979-ba79-4cfe4eecfa26 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:08.213034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac1bce-7763-4979-ba79-4cfe4eecfa26 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:08.213509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac1bce-7763-4979-ba79-4cfe4eecfa26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:08.239407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-60ac1bce-7763-4979-ba79-4cfe4eecfa26 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:08.241138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-60ac1bce-7763-4979-ba79-4cfe4eecfa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:08.253684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fad41-d0fb-45ee-835f-15fe543988fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:08.254738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fad41-d0fb-45ee-835f-15fe543988fa None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:08.255349 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fad41-d0fb-45ee-835f-15fe543988fa None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:08.255720 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fad41-d0fb-45ee-835f-15fe543988fa None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:08.256496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fad41-d0fb-45ee-835f-15fe543988fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:08.258121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fad41-d0fb-45ee-835f-15fe543988fa None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:08.282759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 455/908] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:12:08 2022] POST /identity/v3/services => generated 241 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:08.349591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 454/909] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:08.382807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fad41-d0fb-45ee-835f-15fe543988fa None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:08.496153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fad41-d0fb-45ee-835f-15fe543988fa None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:08.530546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32088729-9f94-4028-9cc2-20831c773ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:08.531872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32088729-9f94-4028-9cc2-20831c773ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:08.532170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32088729-9f94-4028-9cc2-20831c773ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:08.532583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32088729-9f94-4028-9cc2-20831c773ec0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:08.533466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32088729-9f94-4028-9cc2-20831c773ec0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:08.535673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32088729-9f94-4028-9cc2-20831c773ec0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-32088729-9f94-4028-9cc2-20831c773ec0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:08.548549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:08.549615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 17:12:08.550522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:08.556364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 456/910] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:12:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:08.556496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac1bce-7763-4979-ba79-4cfe4eecfa26 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:08.649804 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32088729-9f94-4028-9cc2-20831c773ec0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:08.662723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac1bce-7763-4979-ba79-4cfe4eecfa26 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:08.674333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:08.680228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:08.703272 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddeeba-3ee1-489a-8bed-44d2fd799b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:08.704071 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddeeba-3ee1-489a-8bed-44d2fd799b4d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:08.704348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddeeba-3ee1-489a-8bed-44d2fd799b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:08.704614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ddeeba-3ee1-489a-8bed-44d2fd799b4d None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:08.705353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ddeeba-3ee1-489a-8bed-44d2fd799b4d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:08.705657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ddeeba-3ee1-489a-8bed-44d2fd799b4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:08.707110 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ddeeba-3ee1-489a-8bed-44d2fd799b4d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:08.717254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 455/911] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:12:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:08.718472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddeeba-3ee1-489a-8bed-44d2fd799b4d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:08.770568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32088729-9f94-4028-9cc2-20831c773ec0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:08.805353 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095407e8-34aa-46a2-8510-7b66677ca47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:08.806097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095407e8-34aa-46a2-8510-7b66677ca47c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:08.806507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095407e8-34aa-46a2-8510-7b66677ca47c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:08.806860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095407e8-34aa-46a2-8510-7b66677ca47c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:08.817790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095407e8-34aa-46a2-8510-7b66677ca47c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-095407e8-34aa-46a2-8510-7b66677ca47c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:12:08.822633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:08.824555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:08.826597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:08.826597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:08.826597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:08.826597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:08.826597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:12:08.826597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:12:08.826597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:12:08.826597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:12:08.826597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:12:08.826597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:12:08.826597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 17:12:08.826597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:08.826597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 457/912] 10.222.0.55 () {56 vars in 1120 bytes} [Tue Sep 27 17:12:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:08.833692 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095407e8-34aa-46a2-8510-7b66677ca47c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:08.833908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ddeeba-3ee1-489a-8bed-44d2fd799b4d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:08.875874 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23832c8d-4e3c-4b13-b412-9654df1a2767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:08.876483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23832c8d-4e3c-4b13-b412-9654df1a2767 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:08.876757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23832c8d-4e3c-4b13-b412-9654df1a2767 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:08.877023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23832c8d-4e3c-4b13-b412-9654df1a2767 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:08.877654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23832c8d-4e3c-4b13-b412-9654df1a2767 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:08.877952 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23832c8d-4e3c-4b13-b412-9654df1a2767 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:08.879349 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23832c8d-4e3c-4b13-b412-9654df1a2767 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:08.890780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 456/913] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:12:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:08.893876 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23832c8d-4e3c-4b13-b412-9654df1a2767 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:08.934414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:08.948136 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:08.960222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095407e8-34aa-46a2-8510-7b66677ca47c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:09.004050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21499d6a-5c22-42b2-b220-24c0d594d668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:09.004687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21499d6a-5c22-42b2-b220-24c0d594d668 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:09.004960 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21499d6a-5c22-42b2-b220-24c0d594d668 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:09.005178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21499d6a-5c22-42b2-b220-24c0d594d668 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:09.005843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21499d6a-5c22-42b2-b220-24c0d594d668 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:09.006180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21499d6a-5c22-42b2-b220-24c0d594d668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:09.007545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21499d6a-5c22-42b2-b220-24c0d594d668 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:09.021266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 458/914] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Sep 27 17:12:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:09.022348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21499d6a-5c22-42b2-b220-24c0d594d668 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:09.036029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23832c8d-4e3c-4b13-b412-9654df1a2767 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:09.081630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488e61b-2fa1-4de0-9d2c-1232372ea641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:09.082876 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488e61b-2fa1-4de0-9d2c-1232372ea641 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:09.083325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488e61b-2fa1-4de0-9d2c-1232372ea641 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:09.084007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488e61b-2fa1-4de0-9d2c-1232372ea641 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:09.084864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488e61b-2fa1-4de0-9d2c-1232372ea641 None admin] RBAC: Adding query filter params (interface=public, service_id=8f290c43736f4a1baa35b8b39a4805c5, region_id=RegionOne) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:09.085458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488e61b-2fa1-4de0-9d2c-1232372ea641 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:09.087633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488e61b-2fa1-4de0-9d2c-1232372ea641 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:09.099071 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 457/915] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Sep 27 17:12:08 2022] GET /identity/v3/endpoints?service_id=8f290c43736f4a1baa35b8b39a4805c5&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:09.100179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488e61b-2fa1-4de0-9d2c-1232372ea641 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:09.135535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21499d6a-5c22-42b2-b220-24c0d594d668 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:09.174064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443b9c7-7eb3-41ec-aa06-c4faba274c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:09.174744 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443b9c7-7eb3-41ec-aa06-c4faba274c80 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:09.175021 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443b9c7-7eb3-41ec-aa06-c4faba274c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:09.175292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443b9c7-7eb3-41ec-aa06-c4faba274c80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:09.188142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443b9c7-7eb3-41ec-aa06-c4faba274c80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:09.189661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443b9c7-7eb3-41ec-aa06-c4faba274c80 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:09.191522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c443b9c7-7eb3-41ec-aa06-c4faba274c80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:09.206558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:09.207555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:09.208471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:09.208471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:09.208471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:09.212654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 459/916] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:09.213492 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443b9c7-7eb3-41ec-aa06-c4faba274c80 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:09.251892 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488e61b-2fa1-4de0-9d2c-1232372ea641 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:09.254199 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:09.295635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae5dc0-a1ef-4b05-b67c-42f02320658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:09.296700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae5dc0-a1ef-4b05-b67c-42f02320658c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:09.296980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae5dc0-a1ef-4b05-b67c-42f02320658c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:09.297247 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae5dc0-a1ef-4b05-b67c-42f02320658c None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:09.297900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae5dc0-a1ef-4b05-b67c-42f02320658c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:09.299386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae5dc0-a1ef-4b05-b67c-42f02320658c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:09.314109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 458/917] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Sep 27 17:12:09 2022] GET /identity/v3/services => generated 1497 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:12:09.348483 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443b9c7-7eb3-41ec-aa06-c4faba274c80 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:09.388088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f8f85-fe79-4a51-aa8e-c5ea27a5f95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:09.389108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f8f85-fe79-4a51-aa8e-c5ea27a5f95c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:09.389568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f8f85-fe79-4a51-aa8e-c5ea27a5f95c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:09.390624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f8f85-fe79-4a51-aa8e-c5ea27a5f95c None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:09.391479 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f8f85-fe79-4a51-aa8e-c5ea27a5f95c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:09.391985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f8f85-fe79-4a51-aa8e-c5ea27a5f95c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:09.393582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f8f85-fe79-4a51-aa8e-c5ea27a5f95c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:09.408760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 460/918] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:09.415036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae5dc0-a1ef-4b05-b67c-42f02320658c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:09.531118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae5dc0-a1ef-4b05-b67c-42f02320658c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:09.566114 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4cc74-68b7-48af-a5cd-6a0ee93831b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:09.566661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4cc74-68b7-48af-a5cd-6a0ee93831b8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:09.566936 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4cc74-68b7-48af-a5cd-6a0ee93831b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:09.567200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4cc74-68b7-48af-a5cd-6a0ee93831b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:09.574948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4cc74-68b7-48af-a5cd-6a0ee93831b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:09.576394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4cc74-68b7-48af-a5cd-6a0ee93831b8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-b0c4cc74-68b7-48af-a5cd-6a0ee93831b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:12:09.587893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:09.588934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:09.589923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:09.589923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:09.589923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:12:09.589923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:12:09.589923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:12:09.589923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:12:09.589923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:12:09.589923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:12:09.589923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 17:12:09.589923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:09.589923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 459/919] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Sep 27 17:12:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:09.597580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f8f85-fe79-4a51-aa8e-c5ea27a5f95c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:09.657533 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:09.704436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f8f85-fe79-4a51-aa8e-c5ea27a5f95c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:09.739955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f20e9-8639-4100-9a8e-5123ca71ea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:09.740777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f20e9-8639-4100-9a8e-5123ca71ea3e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:09.741232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f20e9-8639-4100-9a8e-5123ca71ea3e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:09.741658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f20e9-8639-4100-9a8e-5123ca71ea3e None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:09.742615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f20e9-8639-4100-9a8e-5123ca71ea3e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:09.743074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f20e9-8639-4100-9a8e-5123ca71ea3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:09.744632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f20e9-8639-4100-9a8e-5123ca71ea3e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:09.762753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 461/920] 10.222.0.55 () {56 vars in 1149 bytes} [Tue Sep 27 17:12:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:09.768900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4cc74-68b7-48af-a5cd-6a0ee93831b8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:09.798494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:09.876295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4cc74-68b7-48af-a5cd-6a0ee93831b8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:09.915390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c23f8-49a0-4f9d-9aae-d3d660a35aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:09.916025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c23f8-49a0-4f9d-9aae-d3d660a35aba None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:09.916301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c23f8-49a0-4f9d-9aae-d3d660a35aba None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:09.916568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c23f8-49a0-4f9d-9aae-d3d660a35aba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:09.928296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c23f8-49a0-4f9d-9aae-d3d660a35aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:09.929749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c23f8-49a0-4f9d-9aae-d3d660a35aba None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-0b1c23f8-49a0-4f9d-9aae-d3d660a35aba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:09.944339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:09.945346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:09.946292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:09.946292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:09.946292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:09.946292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 460/921] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:09.953318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f20e9-8639-4100-9a8e-5123ca71ea3e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:10.057443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f20e9-8639-4100-9a8e-5123ca71ea3e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:10.072265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:10.095320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be6074-e8ae-4a01-874c-72d3616b0859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:10.095929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be6074-e8ae-4a01-874c-72d3616b0859 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:10.096205 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be6074-e8ae-4a01-874c-72d3616b0859 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:10.096473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be6074-e8ae-4a01-874c-72d3616b0859 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:10.097118 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be6074-e8ae-4a01-874c-72d3616b0859 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:10.097416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be6074-e8ae-4a01-874c-72d3616b0859 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:10.098817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be6074-e8ae-4a01-874c-72d3616b0859 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:10.112503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 462/922] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:10.118182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c23f8-49a0-4f9d-9aae-d3d660a35aba None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:10.200643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:10.226110 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c23f8-49a0-4f9d-9aae-d3d660a35aba None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:10.260794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e652f9-e67b-4185-a090-5c8d4512fe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:10.261421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e652f9-e67b-4185-a090-5c8d4512fe78 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:10.261696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e652f9-e67b-4185-a090-5c8d4512fe78 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:10.261966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e652f9-e67b-4185-a090-5c8d4512fe78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:10.269953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e652f9-e67b-4185-a090-5c8d4512fe78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:10.271425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e652f9-e67b-4185-a090-5c8d4512fe78 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-73e652f9-e67b-4185-a090-5c8d4512fe78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:12:10.282864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:10.283784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:10.285145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:12:10.285145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:12:10.285145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:12:10.285145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:12:10.285145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:12:10.285145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:12:10.285145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:10.285145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:10.285145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 461/923] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:12:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:10.290316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be6074-e8ae-4a01-874c-72d3616b0859 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:10.414365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be6074-e8ae-4a01-874c-72d3616b0859 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:10.453028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a47390-6557-406b-98fd-7dae18d86a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:10.453861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a47390-6557-406b-98fd-7dae18d86a2c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:10.454315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a47390-6557-406b-98fd-7dae18d86a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:10.454749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a47390-6557-406b-98fd-7dae18d86a2c None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:10.455586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a47390-6557-406b-98fd-7dae18d86a2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:10.456051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a47390-6557-406b-98fd-7dae18d86a2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:10.457627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a47390-6557-406b-98fd-7dae18d86a2c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:10.470995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 463/924] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:12:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:10.477105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e652f9-e67b-4185-a090-5c8d4512fe78 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:10.503064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d18e3-5ba8-439f-8086-d6a0036df0f8 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:10.503351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d18e3-5ba8-439f-8086-d6a0036df0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:10.503619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d18e3-5ba8-439f-8086-d6a0036df0f8 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:10.505307 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 464/925] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:10.511672 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b735f34-5901-4551-b03e-86fea55d4641 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:10.511960 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b735f34-5901-4551-b03e-86fea55d4641 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:10.512236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b735f34-5901-4551-b03e-86fea55d4641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:10.535154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1b735f34-5901-4551-b03e-86fea55d4641 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:10.536350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1b735f34-5901-4551-b03e-86fea55d4641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:10.566350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:10.589277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e652f9-e67b-4185-a090-5c8d4512fe78 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:10.614143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 465/926] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:10.614143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4223596-cc8b-4bce-8fb9-ff7d459023ae None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:10.614394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4223596-cc8b-4bce-8fb9-ff7d459023ae None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:10.614627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4223596-cc8b-4bce-8fb9-ff7d459023ae None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:10.616158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 466/927] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:10.623429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7917bfc-1d36-416a-a248-d35680d61e29 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:10.623771 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7917bfc-1d36-416a-a248-d35680d61e29 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:10.624122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7917bfc-1d36-416a-a248-d35680d61e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:10.632565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e175502-4d5c-4773-a1c9-a89e310a6e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:10.633460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e175502-4d5c-4773-a1c9-a89e310a6e7b None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:10.633935 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e175502-4d5c-4773-a1c9-a89e310a6e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:10.634421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e175502-4d5c-4773-a1c9-a89e310a6e7b None admin] PATH_INFO: `/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/c8c3fcd3789c40c88a4c32fff2678fc9/roles/a2cb336de6ca492ab46d7dbad02e31a0` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:10.636840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e175502-4d5c-4773-a1c9-a89e310a6e7b None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1f2574a4344711a56722bca143ae6c, user_id=c8c3fcd3789c40c88a4c32fff2678fc9, role_id=a2cb336de6ca492ab46d7dbad02e31a0)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:10.638694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e175502-4d5c-4773-a1c9-a89e310a6e7b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:10.646620 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a7917bfc-1d36-416a-a248-d35680d61e29 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:10.647940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a7917bfc-1d36-416a-a248-d35680d61e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:10.650787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:10.662899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 462/928] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:12:10 2022] PUT /identity/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/c8c3fcd3789c40c88a4c32fff2678fc9/roles/a2cb336de6ca492ab46d7dbad02e31a0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:12:10.727510 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 467/929] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:10.896551 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e175502-4d5c-4773-a1c9-a89e310a6e7b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:11.004861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e175502-4d5c-4773-a1c9-a89e310a6e7b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:11.008999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7917bfc-1d36-416a-a248-d35680d61e29 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:11.053063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac0b8a-37af-421c-bb19-ed0f5ba281aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:11.053702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac0b8a-37af-421c-bb19-ed0f5ba281aa None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:11.053977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac0b8a-37af-421c-bb19-ed0f5ba281aa None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:11.054285 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac0b8a-37af-421c-bb19-ed0f5ba281aa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:11.054873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac0b8a-37af-421c-bb19-ed0f5ba281aa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:11.056237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac0b8a-37af-421c-bb19-ed0f5ba281aa None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-57ac0b8a-37af-421c-bb19-ed0f5ba281aa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:11.066100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:11.067090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 27 17:12:11.067973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:11.072221 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 463/930] 10.222.0.55 () {56 vars in 1132 bytes} [Tue Sep 27 17:12:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:11.073522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac0b8a-37af-421c-bb19-ed0f5ba281aa None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:11.120974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:11.133656 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7917bfc-1d36-416a-a248-d35680d61e29 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:11.143504 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:11.185171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c63fbc-8a1c-4790-90a8-fb0673619d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:11.186081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c63fbc-8a1c-4790-90a8-fb0673619d13 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:11.186217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac0b8a-37af-421c-bb19-ed0f5ba281aa None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:11.187128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c63fbc-8a1c-4790-90a8-fb0673619d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:11.187548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c63fbc-8a1c-4790-90a8-fb0673619d13 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:11.188541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c63fbc-8a1c-4790-90a8-fb0673619d13 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:11.190977 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c63fbc-8a1c-4790-90a8-fb0673619d13 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-e4c63fbc-8a1c-4790-90a8-fb0673619d13 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:11.202896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:11.204189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:11.205219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:11.208202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 468/931] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Sep 27 17:12:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:11.214003 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c63fbc-8a1c-4790-90a8-fb0673619d13 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:11.229660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fac7b-004a-46f8-be46-2c31c3eac61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:11.230729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fac7b-004a-46f8-be46-2c31c3eac61b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:11.231033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fac7b-004a-46f8-be46-2c31c3eac61b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:11.231390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fac7b-004a-46f8-be46-2c31c3eac61b None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:11.232032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fac7b-004a-46f8-be46-2c31c3eac61b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:11.232329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fac7b-004a-46f8-be46-2c31c3eac61b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:11.234105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fac7b-004a-46f8-be46-2c31c3eac61b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:11.244906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 464/932] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Sep 27 17:12:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:11.246638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fac7b-004a-46f8-be46-2c31c3eac61b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:11.341363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c63fbc-8a1c-4790-90a8-fb0673619d13 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:11.352583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fac7b-004a-46f8-be46-2c31c3eac61b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:11.382543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:11.390742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba0673-fe9d-47ee-a59b-92e3ec5d6d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:11.391354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba0673-fe9d-47ee-a59b-92e3ec5d6d12 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:11.391625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba0673-fe9d-47ee-a59b-92e3ec5d6d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:11.391886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ba0673-fe9d-47ee-a59b-92e3ec5d6d12 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:11.392511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba0673-fe9d-47ee-a59b-92e3ec5d6d12 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:11.392805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba0673-fe9d-47ee-a59b-92e3ec5d6d12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:11.394384 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ba0673-fe9d-47ee-a59b-92e3ec5d6d12 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:11.395639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a7512-3bbd-4920-a8c8-937cabd90526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:11.396461 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a7512-3bbd-4920-a8c8-937cabd90526 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:11.396938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a7512-3bbd-4920-a8c8-937cabd90526 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:11.397417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a7512-3bbd-4920-a8c8-937cabd90526 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:11.398291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a7512-3bbd-4920-a8c8-937cabd90526 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:11.398808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a7512-3bbd-4920-a8c8-937cabd90526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:11.403941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 469/933] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Sep 27 17:12:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:11.404093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a7512-3bbd-4920-a8c8-937cabd90526 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:11.406111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba0673-fe9d-47ee-a59b-92e3ec5d6d12 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:11.414096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 465/934] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Sep 27 17:12:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:11.419053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a7512-3bbd-4920-a8c8-937cabd90526 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:11.436366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:11.538840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a7512-3bbd-4920-a8c8-937cabd90526 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:11.544761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ba0673-fe9d-47ee-a59b-92e3ec5d6d12 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:11.581473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d63be4-286e-4007-8bea-74de206ec307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:11.582021 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d63be4-286e-4007-8bea-74de206ec307 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:11.582318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d63be4-286e-4007-8bea-74de206ec307 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:11.582582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d63be4-286e-4007-8bea-74de206ec307 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:11.583207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d63be4-286e-4007-8bea-74de206ec307 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:11.583501 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d63be4-286e-4007-8bea-74de206ec307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:11.584847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d63be4-286e-4007-8bea-74de206ec307 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:11.587696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83489086-98b6-416c-8ea8-3f17a9126bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:11.588432 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83489086-98b6-416c-8ea8-3f17a9126bbe None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:11.588729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83489086-98b6-416c-8ea8-3f17a9126bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:11.589095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83489086-98b6-416c-8ea8-3f17a9126bbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:11.589767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83489086-98b6-416c-8ea8-3f17a9126bbe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:11.591148 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83489086-98b6-416c-8ea8-3f17a9126bbe None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:11.596551 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 470/935] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Sep 27 17:12:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:11.598789 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d63be4-286e-4007-8bea-74de206ec307 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:11.626806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 466/936] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:12:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:11.626806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce9cff-9890-437b-95bd-78982b559891 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:11.626806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce9cff-9890-437b-95bd-78982b559891 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:11.626806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce9cff-9890-437b-95bd-78982b559891 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:11.626806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 467/937] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:11.632102 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3cb3-cd25-4e8f-96c7-cbea82daca95 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:11.632395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3cb3-cd25-4e8f-96c7-cbea82daca95 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:11.632632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3cb3-cd25-4e8f-96c7-cbea82daca95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:11.654926 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-08ef3cb3-cd25-4e8f-96c7-cbea82daca95 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:11.656890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-08ef3cb3-cd25-4e8f-96c7-cbea82daca95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:11.705369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:11.714918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:11.736281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d63be4-286e-4007-8bea-74de206ec307 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:11.744403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 468/938] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:11.780143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f5911-7081-49bd-b124-ed2a04193ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:11.781290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f5911-7081-49bd-b124-ed2a04193ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:11.781570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f5911-7081-49bd-b124-ed2a04193ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:11.781835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f5911-7081-49bd-b124-ed2a04193ce1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:11.782570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f5911-7081-49bd-b124-ed2a04193ce1 None admin] RBAC: Adding query filter params (interface=public, service_id=4ea495d6b3c845538ac98c6bc4d170bc, region_id=RegionOne) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:11.782872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f5911-7081-49bd-b124-ed2a04193ce1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:11.784303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f5911-7081-49bd-b124-ed2a04193ce1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:11.790679 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 471/939] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Sep 27 17:12:11 2022] GET /identity/v3/endpoints?service_id=4ea495d6b3c845538ac98c6bc4d170bc&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:11.797386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef3cb3-cd25-4e8f-96c7-cbea82daca95 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:11.906667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef3cb3-cd25-4e8f-96c7-cbea82daca95 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:11.942966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b94678-e9ec-422c-a08c-f29aa500c490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:11.943623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b94678-e9ec-422c-a08c-f29aa500c490 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:11.943897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b94678-e9ec-422c-a08c-f29aa500c490 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:11.944168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b94678-e9ec-422c-a08c-f29aa500c490 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:11.944815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b94678-e9ec-422c-a08c-f29aa500c490 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:11.946275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b94678-e9ec-422c-a08c-f29aa500c490 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:11.953283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 469/940] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Sep 27 17:12:11 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:12:12.031935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f5911-7081-49bd-b124-ed2a04193ce1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:12.142211 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f5911-7081-49bd-b124-ed2a04193ce1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:12.176066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02a6f3-a624-4a62-b645-466163945150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:12.176616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02a6f3-a624-4a62-b645-466163945150 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:12.176890 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02a6f3-a624-4a62-b645-466163945150 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:12.177154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02a6f3-a624-4a62-b645-466163945150 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:12.177729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02a6f3-a624-4a62-b645-466163945150 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:12.179108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02a6f3-a624-4a62-b645-466163945150 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1a02a6f3-a624-4a62-b645-466163945150 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:12.188490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:12.189487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:12.190750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 472/941] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:12:12 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:12.195743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b94678-e9ec-422c-a08c-f29aa500c490 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:12.232472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:12.303486 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b94678-e9ec-422c-a08c-f29aa500c490 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:12.341937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3143839-fb0c-40a1-92e8-85edb784d521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:12.342578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3143839-fb0c-40a1-92e8-85edb784d521 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:12.342924 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3143839-fb0c-40a1-92e8-85edb784d521 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:12.343690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3143839-fb0c-40a1-92e8-85edb784d521 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:12.344322 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3143839-fb0c-40a1-92e8-85edb784d521 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:12.344620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3143839-fb0c-40a1-92e8-85edb784d521 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:12.345982 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3143839-fb0c-40a1-92e8-85edb784d521 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:12.351308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 470/942] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:12:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:12.356836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02a6f3-a624-4a62-b645-466163945150 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:12.372141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:12.466794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02a6f3-a624-4a62-b645-466163945150 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:12.504556 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c2b7f-466b-4495-b1c2-09a97daab8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:12.505157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c2b7f-466b-4495-b1c2-09a97daab8ca None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:12.505431 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c2b7f-466b-4495-b1c2-09a97daab8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:12.505699 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c2b7f-466b-4495-b1c2-09a97daab8ca None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:12.506354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c2b7f-466b-4495-b1c2-09a97daab8ca None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:12.506650 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c2b7f-466b-4495-b1c2-09a97daab8ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:12.508005 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c2b7f-466b-4495-b1c2-09a97daab8ca None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:12.513273 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 473/943] 10.222.0.55 () {56 vars in 1114 bytes} [Tue Sep 27 17:12:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:12.519438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3143839-fb0c-40a1-92e8-85edb784d521 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:12.621609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:12.624381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3143839-fb0c-40a1-92e8-85edb784d521 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:12.666184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200c580-dede-413b-816f-248848b19492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:12.666797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c580-dede-413b-816f-248848b19492 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:12.667071 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c580-dede-413b-816f-248848b19492 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:12.667343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200c580-dede-413b-816f-248848b19492 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:12.668025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200c580-dede-413b-816f-248848b19492 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:12.669396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200c580-dede-413b-816f-248848b19492 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:12.694112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 471/944] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:12:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:12.987721 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451abad0-f427-4385-af74-9b0a2977325b None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:12.988363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451abad0-f427-4385-af74-9b0a2977325b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:12.988858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451abad0-f427-4385-af74-9b0a2977325b None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:12.990739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 474/945] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:12.996255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78470c-4922-4e73-a739-d7ccdba79613 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:12.996771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78470c-4922-4e73-a739-d7ccdba79613 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:12.997247 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78470c-4922-4e73-a739-d7ccdba79613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:13.016475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3d78470c-4922-4e73-a739-d7ccdba79613 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:13.018011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3d78470c-4922-4e73-a739-d7ccdba79613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:13.098176 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 472/946] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:13.369404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451abad0-f427-4385-af74-9b0a2977325b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:13.450126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:13.474636 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451abad0-f427-4385-af74-9b0a2977325b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:13.513386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d3f3a-c2d0-48b2-8a9f-7a940cd19230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:13.513999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d3f3a-c2d0-48b2-8a9f-7a940cd19230 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:13.514296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d3f3a-c2d0-48b2-8a9f-7a940cd19230 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:13.514559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d3f3a-c2d0-48b2-8a9f-7a940cd19230 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:13.525927 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d3f3a-c2d0-48b2-8a9f-7a940cd19230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-8c5d3f3a-c2d0-48b2-8a9f-7a940cd19230 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:12:13.530675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:13.531621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:13.532495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:13.532495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:13.532495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:13.532495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:13.532495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:12:13.532495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:12:13.532495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:12:13.532495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:12:13.532495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:12:13.532495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:12:13.532495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 17:12:13.532495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:13.532495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 475/947] 10.222.0.55 () {56 vars in 1120 bytes} [Tue Sep 27 17:12:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:13.538617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78470c-4922-4e73-a739-d7ccdba79613 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:13.626040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:13.651324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78470c-4922-4e73-a739-d7ccdba79613 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:13.688047 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18457037-d79b-4baa-bdcd-b94aefa71617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:13.688659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18457037-d79b-4baa-bdcd-b94aefa71617 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:13.688936 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18457037-d79b-4baa-bdcd-b94aefa71617 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:13.689206 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18457037-d79b-4baa-bdcd-b94aefa71617 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:13.689934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18457037-d79b-4baa-bdcd-b94aefa71617 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:13.690268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18457037-d79b-4baa-bdcd-b94aefa71617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:13.691640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18457037-d79b-4baa-bdcd-b94aefa71617 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:13.704845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 473/948] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Sep 27 17:12:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:13.706292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d3f3a-c2d0-48b2-8a9f-7a940cd19230 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:13.809051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d3f3a-c2d0-48b2-8a9f-7a940cd19230 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:13.843744 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2599a3-09d0-4a72-b204-c77e4fb76913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:13.844374 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2599a3-09d0-4a72-b204-c77e4fb76913 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:13.844653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2599a3-09d0-4a72-b204-c77e4fb76913 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:13.844923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2599a3-09d0-4a72-b204-c77e4fb76913 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:13.847625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:13.856714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2599a3-09d0-4a72-b204-c77e4fb76913 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:13.858191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2599a3-09d0-4a72-b204-c77e4fb76913 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-9e2599a3-09d0-4a72-b204-c77e4fb76913 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:13.873635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:13.874791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:13.875689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:13.875689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:13.875689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:13.879975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 476/949] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:13.881469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18457037-d79b-4baa-bdcd-b94aefa71617 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:14.003704 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18457037-d79b-4baa-bdcd-b94aefa71617 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:14.018707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e40d6-9ed9-4364-8670-06ea44893e06 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:14.019100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e40d6-9ed9-4364-8670-06ea44893e06 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:14.019270 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e40d6-9ed9-4364-8670-06ea44893e06 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:14.020688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 477/950] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:14.026295 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa876f70-18de-48fb-a7bb-563f5670850f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:14.026837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa876f70-18de-48fb-a7bb-563f5670850f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:14.027312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa876f70-18de-48fb-a7bb-563f5670850f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:14.048665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb6aa3-d9d7-488c-a58a-45bac5ed83ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:14.048866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fa876f70-18de-48fb-a7bb-563f5670850f None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:14.049738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb6aa3-d9d7-488c-a58a-45bac5ed83ab None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:14.049985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fa876f70-18de-48fb-a7bb-563f5670850f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:14.050482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb6aa3-d9d7-488c-a58a-45bac5ed83ab None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:14.050949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb6aa3-d9d7-488c-a58a-45bac5ed83ab None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:14.051923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb6aa3-d9d7-488c-a58a-45bac5ed83ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:14.052424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb6aa3-d9d7-488c-a58a-45bac5ed83ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:14.054190 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb6aa3-d9d7-488c-a58a-45bac5ed83ab None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:14.067604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 474/951] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:14.073096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb6aa3-d9d7-488c-a58a-45bac5ed83ab None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:14.078025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:14.143142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 478/952] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:14.197415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb6aa3-d9d7-488c-a58a-45bac5ed83ab None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:14.232772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c7b71-6659-4abb-b9e2-5ef1104fb47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:14.233482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c7b71-6659-4abb-b9e2-5ef1104fb47c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:14.233759 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c7b71-6659-4abb-b9e2-5ef1104fb47c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:14.234023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c7b71-6659-4abb-b9e2-5ef1104fb47c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:14.241317 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c7b71-6659-4abb-b9e2-5ef1104fb47c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:14.242832 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8c7b71-6659-4abb-b9e2-5ef1104fb47c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:14.245341 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f5967-2044-487e-808a-05026bde213a None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:14.245635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f5967-2044-487e-808a-05026bde213a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:14.245910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f5967-2044-487e-808a-05026bde213a None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:14.247492 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 479/953] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:14.252931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7320244-1232-404e-97d6-77670b22c549 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:14.253210 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7320244-1232-404e-97d6-77670b22c549 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:14.253484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7320244-1232-404e-97d6-77670b22c549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ff8c7b71-6659-4abb-b9e2-5ef1104fb47c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:12:14.255412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:14.256340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:14.257304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:14.257304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:14.257304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:12:14.257304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:12:14.257304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:12:14.257304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:12:14.257304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:12:14.257304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:12:14.257304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 27 17:12:14.257304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:14.261735 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 475/954] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Sep 27 17:12:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:14.262887 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c7b71-6659-4abb-b9e2-5ef1104fb47c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:14.273299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e7320244-1232-404e-97d6-77670b22c549 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:14.274717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e7320244-1232-404e-97d6-77670b22c549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:14.342678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:14.368664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 480/955] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:14.387099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c7b71-6659-4abb-b9e2-5ef1104fb47c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:14.424164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2242f-fd67-4759-9e10-2b9cda25f7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:14.425078 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2242f-fd67-4759-9e10-2b9cda25f7bf None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:14.425572 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2242f-fd67-4759-9e10-2b9cda25f7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:14.426037 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2242f-fd67-4759-9e10-2b9cda25f7bf None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:14.427043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a2242f-fd67-4759-9e10-2b9cda25f7bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:14.427607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a2242f-fd67-4759-9e10-2b9cda25f7bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:14.429325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a2242f-fd67-4759-9e10-2b9cda25f7bf None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:14.430340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7320244-1232-404e-97d6-77670b22c549 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:14.454343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 476/956] 10.222.0.55 () {56 vars in 1149 bytes} [Tue Sep 27 17:12:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:14.455873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2242f-fd67-4759-9e10-2b9cda25f7bf None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:14.494484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:14.562987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7320244-1232-404e-97d6-77670b22c549 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:14.567907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a2242f-fd67-4759-9e10-2b9cda25f7bf None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:14.592782 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:14.607107 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a292d-529a-4180-bd71-fefdb4dbb620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:14.607689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def74766-f092-4325-a2e6-3eb89497fc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:14.607808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a292d-529a-4180-bd71-fefdb4dbb620 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:14.608111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a292d-529a-4180-bd71-fefdb4dbb620 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:14.608785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a292d-529a-4180-bd71-fefdb4dbb620 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:14.609359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a292d-529a-4180-bd71-fefdb4dbb620 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:14.609989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def74766-f092-4325-a2e6-3eb89497fc77 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:14.610761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a292d-529a-4180-bd71-fefdb4dbb620 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:14.611062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def74766-f092-4325-a2e6-3eb89497fc77 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:14.611736 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def74766-f092-4325-a2e6-3eb89497fc77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-987a292d-529a-4180-bd71-fefdb4dbb620 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:14.621028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:14.622275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:14.628310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 481/957] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:12:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:14.630988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a292d-529a-4180-bd71-fefdb4dbb620 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:14.690863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def74766-f092-4325-a2e6-3eb89497fc77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:14.692219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def74766-f092-4325-a2e6-3eb89497fc77 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:14.738384 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a292d-529a-4180-bd71-fefdb4dbb620 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:14.752615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:14.777238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27146dbc-54fa-4bfd-871c-4a4bd6356e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:14.777836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27146dbc-54fa-4bfd-871c-4a4bd6356e61 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:14.778140 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27146dbc-54fa-4bfd-871c-4a4bd6356e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:14.778412 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27146dbc-54fa-4bfd-871c-4a4bd6356e61 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:14.779096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27146dbc-54fa-4bfd-871c-4a4bd6356e61 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:14.779392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27146dbc-54fa-4bfd-871c-4a4bd6356e61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:14.780856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27146dbc-54fa-4bfd-871c-4a4bd6356e61 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:14.786036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 482/958] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Sep 27 17:12:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:14.786908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27146dbc-54fa-4bfd-871c-4a4bd6356e61 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:14.797644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 477/959] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:12:14 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 27 17:12:14.798502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def74766-f092-4325-a2e6-3eb89497fc77 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:14.830523 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:14.899322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27146dbc-54fa-4bfd-871c-4a4bd6356e61 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:14.914557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def74766-f092-4325-a2e6-3eb89497fc77 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:14.937324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f0617-0cd4-440a-9514-1d4545671a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:14.937949 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f0617-0cd4-440a-9514-1d4545671a76 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:14.938254 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f0617-0cd4-440a-9514-1d4545671a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:14.938545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f0617-0cd4-440a-9514-1d4545671a76 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:14.939201 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f0617-0cd4-440a-9514-1d4545671a76 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:14.940542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f0617-0cd4-440a-9514-1d4545671a76 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-245f0617-0cd4-440a-9514-1d4545671a76 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:14.950119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:14.951174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:14.952135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 483/960] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Sep 27 17:12:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:14.953584 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f0617-0cd4-440a-9514-1d4545671a76 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:14.957660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3602e-95f1-45b2-a214-d20e2dba6a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:14.958327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3602e-95f1-45b2-a214-d20e2dba6a33 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:14.958614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3602e-95f1-45b2-a214-d20e2dba6a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:14.958891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3602e-95f1-45b2-a214-d20e2dba6a33 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:14.959536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3602e-95f1-45b2-a214-d20e2dba6a33 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:14.959843 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3602e-95f1-45b2-a214-d20e2dba6a33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:14.961217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3602e-95f1-45b2-a214-d20e2dba6a33 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:14.974941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 478/961] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Sep 27 17:12:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:14.974941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3602e-95f1-45b2-a214-d20e2dba6a33 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:15.017764 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:15.070199 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:15.086900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3602e-95f1-45b2-a214-d20e2dba6a33 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:15.087223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f0617-0cd4-440a-9514-1d4545671a76 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:15.125842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcd01f-16c4-4049-ac33-b3be567ad104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:15.126414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a30b3-f385-4c66-822a-830d04864e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:15.126533 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd01f-16c4-4049-ac33-b3be567ad104 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:15.126759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd01f-16c4-4049-ac33-b3be567ad104 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:15.127025 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd01f-16c4-4049-ac33-b3be567ad104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:15.127119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a30b3-f385-4c66-822a-830d04864e83 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:15.127299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a30b3-f385-4c66-822a-830d04864e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:15.127570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a30b3-f385-4c66-822a-830d04864e83 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:15.128214 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a30b3-f385-4c66-822a-830d04864e83 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:15.128515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a30b3-f385-4c66-822a-830d04864e83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:15.130197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a30b3-f385-4c66-822a-830d04864e83 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:15.135346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 479/962] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Sep 27 17:12:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:15.139285 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcd01f-16c4-4049-ac33-b3be567ad104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:15.141469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcd01f-16c4-4049-ac33-b3be567ad104 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:15.142225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e4f0b-7a3b-4437-83c7-85952ab47802 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:15.142470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e4f0b-7a3b-4437-83c7-85952ab47802 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:15.142745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e4f0b-7a3b-4437-83c7-85952ab47802 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:15.144051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 480/963] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:15.144747 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e4f0b-7a3b-4437-83c7-85952ab47802 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d8fcd01f-16c4-4049-ac33-b3be567ad104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:15.164054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:15.165212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:15.166368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:15.166368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:15.166368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:15.166579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 484/964] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:15.167787 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26690773-546a-4e60-abfc-9de936d6e7dd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:15.168070 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26690773-546a-4e60-abfc-9de936d6e7dd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:15.168337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26690773-546a-4e60-abfc-9de936d6e7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:15.194495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-26690773-546a-4e60-abfc-9de936d6e7dd None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:15.195669 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-26690773-546a-4e60-abfc-9de936d6e7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:15.255414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e4f0b-7a3b-4437-83c7-85952ab47802 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:15.274103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 485/965] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:15.274103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26690773-546a-4e60-abfc-9de936d6e7dd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:15.291611 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9184e0f-a05b-4f51-a2de-dd0310f560dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:15.292574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9184e0f-a05b-4f51-a2de-dd0310f560dc None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:15.292909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9184e0f-a05b-4f51-a2de-dd0310f560dc None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:15.293239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9184e0f-a05b-4f51-a2de-dd0310f560dc None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:15.294221 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9184e0f-a05b-4f51-a2de-dd0310f560dc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:15.294968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9184e0f-a05b-4f51-a2de-dd0310f560dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:15.296435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9184e0f-a05b-4f51-a2de-dd0310f560dc None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:15.301532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:15.305730 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 481/966] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Sep 27 17:12:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:15.309573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9184e0f-a05b-4f51-a2de-dd0310f560dc None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:15.374486 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:15.406698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26690773-546a-4e60-abfc-9de936d6e7dd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:15.413641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9184e0f-a05b-4f51-a2de-dd0310f560dc None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:15.448196 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d420a-ded4-4fd7-bf2a-4067bf368065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:15.449016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d420a-ded4-4fd7-bf2a-4067bf368065 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:15.449377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d420a-ded4-4fd7-bf2a-4067bf368065 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:15.449744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d420a-ded4-4fd7-bf2a-4067bf368065 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:15.450570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d420a-ded4-4fd7-bf2a-4067bf368065 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:15.451628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40a1ac-b02a-4c2d-a149-232f784c85b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:15.452789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d420a-ded4-4fd7-bf2a-4067bf368065 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:15.453410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a1ac-b02a-4c2d-a149-232f784c85b7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:15.453832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a1ac-b02a-4c2d-a149-232f784c85b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:15.454270 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a1ac-b02a-4c2d-a149-232f784c85b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:15.455079 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40a1ac-b02a-4c2d-a149-232f784c85b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:15.455534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40a1ac-b02a-4c2d-a149-232f784c85b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:15.457027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40a1ac-b02a-4c2d-a149-232f784c85b7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:15.477250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 486/967] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:15.482703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 482/968] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:12:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:15.483961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40a1ac-b02a-4c2d-a149-232f784c85b7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:15.556320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d420a-ded4-4fd7-bf2a-4067bf368065 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:15.588953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:15.621297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40a1ac-b02a-4c2d-a149-232f784c85b7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:15.647837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:15.688075 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36acf6-eaa7-47c9-ba64-6a9d40a0195d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:15.689791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36acf6-eaa7-47c9-ba64-6a9d40a0195d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:15.690085 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36acf6-eaa7-47c9-ba64-6a9d40a0195d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:15.690364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36acf6-eaa7-47c9-ba64-6a9d40a0195d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:15.692615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d420a-ded4-4fd7-bf2a-4067bf368065 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:15.700251 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36acf6-eaa7-47c9-ba64-6a9d40a0195d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:15.702756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36acf6-eaa7-47c9-ba64-6a9d40a0195d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0f36acf6-eaa7-47c9-ba64-6a9d40a0195d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:12:15.716364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:15.717511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:15.718604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:12:15.718604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:12:15.718604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:12:15.718604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:12:15.718604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:12:15.718604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:12:15.718604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:15.718604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:15.718604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 487/969] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:12:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:15.723506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36acf6-eaa7-47c9-ba64-6a9d40a0195d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:15.742117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec02a1-06e6-41dd-ab1d-2c70721c072d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:15.742741 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec02a1-06e6-41dd-ab1d-2c70721c072d None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:15.743017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec02a1-06e6-41dd-ab1d-2c70721c072d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:15.743283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec02a1-06e6-41dd-ab1d-2c70721c072d None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:15.743929 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec02a1-06e6-41dd-ab1d-2c70721c072d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:15.745348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec02a1-06e6-41dd-ab1d-2c70721c072d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:15.747957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: WARNING keystone.server.flask.common [None req-2aec02a1-06e6-41dd-ab1d-2c70721c072d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 27 17:12:15.773021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.sql.core [None req-2aec02a1-06e6-41dd-ab1d-2c70721c072d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 27 17:12:15.773021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 27 17:12:15.773021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [parameters: {'id': '7aa979a69863479ba180b75f7a948b8b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 27 17:12:15.773021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81426) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 27 17:12:15.782987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-2aec02a1-06e6-41dd-ab1d-2c70721c072d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7aa979a69863479ba180b75f7a948b8b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7aa979a69863479ba180b75f7a948b8b. Sep 27 17:12:15.782987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:15.782987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 17:12:15.782987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 17:12:15.782987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 17:12:15.782987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 17:12:15.782987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 17:12:15.782987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 17:12:15.782987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 17:12:15.782987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application conn.query(q) Sep 27 17:12:15.782987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 17:12:15.782987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 17:12:15.782987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application result.read() Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 27 17:12:15.783878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application next(self.gen) Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application yield resource Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application next(self.gen) Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.session.rollback() Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.force_reraise() Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise self.value Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application session.commit() Sep 27 17:12:15.784868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._prepare_impl() Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.session.flush() Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._flush(objects) Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application compat.raise_( Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application flush_context.execute() Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 27 17:12:15.785793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rec.execute(self) Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 27 17:12:15.786739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application conn.query(q) Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application result.read() Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 17:12:15.789151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [parameters: {'id': '7aa979a69863479ba180b75f7a948b8b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:15.791750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:15.793679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:15.795930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:15.795930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 27 17:12:15.795930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 27 17:12:15.795930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:15.795930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:15.795930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 27 17:12:15.795930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 27 17:12:15.795930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7aa979a69863479ba180b75f7a948b8b. Sep 27 17:12:15.795930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:15.795930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 483/970] 10.222.0.55 () {60 vars in 1140 bytes} [Tue Sep 27 17:12:15 2022] POST /identity/v3/projects => generated 262 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:15.795930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec02a1-06e6-41dd-ab1d-2c70721c072d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:15.851589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36acf6-eaa7-47c9-ba64-6a9d40a0195d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:15.890587 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a51287-ff88-43bc-9ff7-f1405746b0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:15.891199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a51287-ff88-43bc-9ff7-f1405746b0bd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:15.891476 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a51287-ff88-43bc-9ff7-f1405746b0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:15.891743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a51287-ff88-43bc-9ff7-f1405746b0bd None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:15.892565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a51287-ff88-43bc-9ff7-f1405746b0bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:15.892754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a51287-ff88-43bc-9ff7-f1405746b0bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:15.894245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a51287-ff88-43bc-9ff7-f1405746b0bd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:15.901818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:15.905679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec02a1-06e6-41dd-ab1d-2c70721c072d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:15.913554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 488/971] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:12:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:15.914708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a51287-ff88-43bc-9ff7-f1405746b0bd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:15.938970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:15.950794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce54d4-c931-4406-8afa-1a7a0bbdf677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:15.951405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce54d4-c931-4406-8afa-1a7a0bbdf677 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:15.951681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce54d4-c931-4406-8afa-1a7a0bbdf677 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:15.951949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce54d4-c931-4406-8afa-1a7a0bbdf677 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:15.965134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce54d4-c931-4406-8afa-1a7a0bbdf677 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:15.968085 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce54d4-c931-4406-8afa-1a7a0bbdf677 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-59ce54d4-c931-4406-8afa-1a7a0bbdf677 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:12:15.984910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:15.985831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:15.986774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:12:15.986774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:12:15.986774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:12:15.986774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:12:15.986774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:12:15.986774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:12:15.986774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:15.986774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:15.993980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 484/972] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:12:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:15.995000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce54d4-c931-4406-8afa-1a7a0bbdf677 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:16.032001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a51287-ff88-43bc-9ff7-f1405746b0bd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:16.071227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace1a1c-ce99-49a5-b860-4c148eaa982d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:16.071837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace1a1c-ce99-49a5-b860-4c148eaa982d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:16.072172 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace1a1c-ce99-49a5-b860-4c148eaa982d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:16.072445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace1a1c-ce99-49a5-b860-4c148eaa982d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:16.073223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace1a1c-ce99-49a5-b860-4c148eaa982d None admin] RBAC: Adding query filter params (role.id=a2cb336de6ca492ab46d7dbad02e31a0, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=c8c3fcd3789c40c88a4c32fff2678fc9) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:16.073535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace1a1c-ce99-49a5-b860-4c148eaa982d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:16.074991 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace1a1c-ce99-49a5-b860-4c148eaa982d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:16.090113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 489/973] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:12:15 2022] GET /identity/v3/role_assignments?user.id=c8c3fcd3789c40c88a4c32fff2678fc9&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=a2cb336de6ca492ab46d7dbad02e31a0 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:16.120167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce54d4-c931-4406-8afa-1a7a0bbdf677 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:16.158733 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dd3ad-89c6-4815-85a7-f1b61a425af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:16.159356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dd3ad-89c6-4815-85a7-f1b61a425af3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:16.159635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dd3ad-89c6-4815-85a7-f1b61a425af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:16.159907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dd3ad-89c6-4815-85a7-f1b61a425af3 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:16.160586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2dd3ad-89c6-4815-85a7-f1b61a425af3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:16.160888 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2dd3ad-89c6-4815-85a7-f1b61a425af3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:16.162546 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2dd3ad-89c6-4815-85a7-f1b61a425af3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:16.186032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 485/974] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Sep 27 17:12:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:17.181090 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdae6d-8b30-4fe6-8354-25a161e05cc3 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:17.181619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdae6d-8b30-4fe6-8354-25a161e05cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:17.181619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdae6d-8b30-4fe6-8354-25a161e05cc3 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:17.183181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 490/975] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:17.189952 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b38d2a-7ee4-4f93-a4d4-e0d9871e15be None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:17.190429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b38d2a-7ee4-4f93-a4d4-e0d9871e15be None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:17.190510 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b38d2a-7ee4-4f93-a4d4-e0d9871e15be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:17.209163 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-46b38d2a-7ee4-4f93-a4d4-e0d9871e15be None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:17.210674 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-46b38d2a-7ee4-4f93-a4d4-e0d9871e15be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:17.254906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:17.281509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 486/976] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:17.558743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdae6d-8b30-4fe6-8354-25a161e05cc3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:17.620852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:17.662141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdae6d-8b30-4fe6-8354-25a161e05cc3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:17.705677 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac55f19-e00a-4e43-90d0-eabee638c5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:17.706311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac55f19-e00a-4e43-90d0-eabee638c5ac None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:17.706590 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac55f19-e00a-4e43-90d0-eabee638c5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:17.706861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac55f19-e00a-4e43-90d0-eabee638c5ac None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:17.707528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac55f19-e00a-4e43-90d0-eabee638c5ac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:17.708944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac55f19-e00a-4e43-90d0-eabee638c5ac None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:17.726149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 491/977] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:12:17 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:17.788303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322f0e5-8b84-4665-b0c8-30a02c4883dd None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:17.788620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322f0e5-8b84-4665-b0c8-30a02c4883dd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:17.788842 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322f0e5-8b84-4665-b0c8-30a02c4883dd None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:17.790506 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 487/978] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:17.796417 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce988ef-40ac-4499-8a45-d5502cea84d7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:17.796895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce988ef-40ac-4499-8a45-d5502cea84d7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:17.797323 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce988ef-40ac-4499-8a45-d5502cea84d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:17.819011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5ce988ef-40ac-4499-8a45-d5502cea84d7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:17.820510 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5ce988ef-40ac-4499-8a45-d5502cea84d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:17.906269 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 492/979] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:18.172915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322f0e5-8b84-4665-b0c8-30a02c4883dd None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:18.280825 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322f0e5-8b84-4665-b0c8-30a02c4883dd None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:18.332148 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14916c-6354-4e4a-a748-c0a95839b72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:18.332787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14916c-6354-4e4a-a748-c0a95839b72c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:18.333064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14916c-6354-4e4a-a748-c0a95839b72c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:18.333335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14916c-6354-4e4a-a748-c0a95839b72c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:18.333928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14916c-6354-4e4a-a748-c0a95839b72c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:18.335336 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14916c-6354-4e4a-a748-c0a95839b72c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-5e14916c-6354-4e4a-a748-c0a95839b72c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:18.345079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:18.346129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:18.347398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 488/980] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Sep 27 17:12:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:18.354506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce988ef-40ac-4499-8a45-d5502cea84d7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:18.407387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:18.435658 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3468ba8-9b1a-4cd2-970a-7e8caf50654a None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:18.435831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3468ba8-9b1a-4cd2-970a-7e8caf50654a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:18.436110 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3468ba8-9b1a-4cd2-970a-7e8caf50654a None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:18.437777 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 489/981] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:18.442981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60869169-0ced-4b25-b275-958106fe79ad None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:18.443250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60869169-0ced-4b25-b275-958106fe79ad None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:18.443475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60869169-0ced-4b25-b275-958106fe79ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:18.462186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-60869169-0ced-4b25-b275-958106fe79ad None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:18.463495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-60869169-0ced-4b25-b275-958106fe79ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:18.468169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce988ef-40ac-4499-8a45-d5502cea84d7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:18.503758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:18.519350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18fa45-ff4f-4bff-b546-21aa0ddcf175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:18.520535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18fa45-ff4f-4bff-b546-21aa0ddcf175 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:18.521250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18fa45-ff4f-4bff-b546-21aa0ddcf175 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:18.521718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18fa45-ff4f-4bff-b546-21aa0ddcf175 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:18.522953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18fa45-ff4f-4bff-b546-21aa0ddcf175 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:18.523452 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18fa45-ff4f-4bff-b546-21aa0ddcf175 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:18.525482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18fa45-ff4f-4bff-b546-21aa0ddcf175 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:18.531734 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 493/982] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Sep 27 17:12:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:18.533288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593cb57-cd3c-48f7-a21e-f48909638619 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:18.533741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593cb57-cd3c-48f7-a21e-f48909638619 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:18.534006 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593cb57-cd3c-48f7-a21e-f48909638619 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:18.535200 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 494/983] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:18.539129 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593cb57-cd3c-48f7-a21e-f48909638619 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:18.544754 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 490/984] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:18.546210 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca87de-aeec-4bbe-92b1-3c225bd5e869 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:18.546490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca87de-aeec-4bbe-92b1-3c225bd5e869 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:18.546758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca87de-aeec-4bbe-92b1-3c225bd5e869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:18.564610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-52ca87de-aeec-4bbe-92b1-3c225bd5e869 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:18.566002 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-52ca87de-aeec-4bbe-92b1-3c225bd5e869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:18.654767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593cb57-cd3c-48f7-a21e-f48909638619 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:18.658203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 491/985] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:18.677387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:18.698971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6097b0-0c37-4db2-b6bc-67c330508388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:18.699501 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6097b0-0c37-4db2-b6bc-67c330508388 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:18.699770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6097b0-0c37-4db2-b6bc-67c330508388 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:18.700034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6097b0-0c37-4db2-b6bc-67c330508388 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:18.700646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6097b0-0c37-4db2-b6bc-67c330508388 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:18.700938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6097b0-0c37-4db2-b6bc-67c330508388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:18.702331 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6097b0-0c37-4db2-b6bc-67c330508388 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:18.707358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 495/986] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Sep 27 17:12:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:18.712934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca87de-aeec-4bbe-92b1-3c225bd5e869 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:18.817375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:18.820816 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca87de-aeec-4bbe-92b1-3c225bd5e869 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:18.849907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6097b0-0c37-4db2-b6bc-67c330508388 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:18.859305 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb400b-dc81-4931-99d9-2691d5665345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:18.859913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb400b-dc81-4931-99d9-2691d5665345 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:18.860191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb400b-dc81-4931-99d9-2691d5665345 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:18.860462 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb400b-dc81-4931-99d9-2691d5665345 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:18.861175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb400b-dc81-4931-99d9-2691d5665345 None admin] RBAC: Adding query filter params (interface=admin, service_id=4ea495d6b3c845538ac98c6bc4d170bc, region_id=RegionOne) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:18.861482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb400b-dc81-4931-99d9-2691d5665345 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:18.862969 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb400b-dc81-4931-99d9-2691d5665345 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:18.868468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 492/987] 10.222.0.55 () {56 vars in 1249 bytes} [Tue Sep 27 17:12:18 2022] GET /identity/v3/endpoints?service_id=4ea495d6b3c845538ac98c6bc4d170bc&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:18.874542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb400b-dc81-4931-99d9-2691d5665345 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:18.952514 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6097b0-0c37-4db2-b6bc-67c330508388 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:18.977910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb400b-dc81-4931-99d9-2691d5665345 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:18.988077 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dc01f-dbf4-46ff-9610-74756f2568d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:18.988702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dc01f-dbf4-46ff-9610-74756f2568d3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:18.988977 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dc01f-dbf4-46ff-9610-74756f2568d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:18.989246 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788dc01f-dbf4-46ff-9610-74756f2568d3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:18.989825 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788dc01f-dbf4-46ff-9610-74756f2568d3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:18.991211 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788dc01f-dbf4-46ff-9610-74756f2568d3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-788dc01f-dbf4-46ff-9610-74756f2568d3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:19.001095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:19.002182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 17:12:19.003229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:19.006211 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 496/988] 10.222.0.55 () {56 vars in 1105 bytes} [Tue Sep 27 17:12:18 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:19.007320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dc01f-dbf4-46ff-9610-74756f2568d3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:19.017638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb1ed7-bc13-4c3c-b4bd-263827dcc884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:19.018366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb1ed7-bc13-4c3c-b4bd-263827dcc884 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:19.018566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb1ed7-bc13-4c3c-b4bd-263827dcc884 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:19.018833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb1ed7-bc13-4c3c-b4bd-263827dcc884 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:19.019429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb1ed7-bc13-4c3c-b4bd-263827dcc884 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:19.020778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb1ed7-bc13-4c3c-b4bd-263827dcc884 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:19.027330 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 493/989] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Sep 27 17:12:18 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:12:19.028076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb1ed7-bc13-4c3c-b4bd-263827dcc884 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:19.060059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:19.084920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:19.144663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb1ed7-bc13-4c3c-b4bd-263827dcc884 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:19.171656 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788dc01f-dbf4-46ff-9610-74756f2568d3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:19.183244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33598d0-58d1-4a40-a4d9-9a5bcf7baf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:19.183995 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33598d0-58d1-4a40-a4d9-9a5bcf7baf02 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:19.184331 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33598d0-58d1-4a40-a4d9-9a5bcf7baf02 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:19.184668 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33598d0-58d1-4a40-a4d9-9a5bcf7baf02 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:19.185574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33598d0-58d1-4a40-a4d9-9a5bcf7baf02 None admin] RBAC: Adding query filter params (name=image) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:19.185814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33598d0-58d1-4a40-a4d9-9a5bcf7baf02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:19.187619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33598d0-58d1-4a40-a4d9-9a5bcf7baf02 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:19.197843 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 494/990] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:19.199177 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33598d0-58d1-4a40-a4d9-9a5bcf7baf02 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:19.223845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c3dbe-2dec-4fc2-8743-955a0328d840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:19.224623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c3dbe-2dec-4fc2-8743-955a0328d840 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:19.224623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c3dbe-2dec-4fc2-8743-955a0328d840 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:19.224902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c3dbe-2dec-4fc2-8743-955a0328d840 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:19.225552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c3dbe-2dec-4fc2-8743-955a0328d840 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:19.227000 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c3dbe-2dec-4fc2-8743-955a0328d840 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:19.229762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: WARNING keystone.server.flask.common [None req-f91c3dbe-2dec-4fc2-8743-955a0328d840 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 27 17:12:19.247195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.sql.core [None req-f91c3dbe-2dec-4fc2-8743-955a0328d840 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 27 17:12:19.247195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 27 17:12:19.247195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [parameters: {'user_id': '865f0eecf080494097ea81060092b394', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 27 17:12:19.247195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81425) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-f91c3dbe-2dec-4fc2-8743-955a0328d840 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application conn.query(q) Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result.read() Sep 27 17:12:19.267122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 17:12:19.268126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application next(self.gen) Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application yield resource Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application next(self.gen) Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.session.rollback() Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.force_reraise() Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise self.value Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application session.commit() Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 27 17:12:19.269102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._prepare_impl() Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.session.flush() Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._flush(objects) Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application compat.raise_( Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application flush_context.execute() Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rec.execute(self) Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 27 17:12:19.270821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 17:12:19.271770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result = self._query(query) Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application conn.query(q) Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result.read() Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 17:12:19.272712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [parameters: {'user_id': '865f0eecf080494097ea81060092b394', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:19.273559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 27 17:12:19.275511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 27 17:12:19.277708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 27 17:12:19.277708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 27 17:12:19.277708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 27 17:12:19.277708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 27 17:12:19.277708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 27 17:12:19.277708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:19.277708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 497/991] 10.222.0.55 () {60 vars in 1131 bytes} [Tue Sep 27 17:12:19 2022] POST /identity/v3/users => generated 160 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:19.277708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c3dbe-2dec-4fc2-8743-955a0328d840 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:19.330811 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:19.341429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33598d0-58d1-4a40-a4d9-9a5bcf7baf02 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:19.377779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede3c41-8cec-4fd3-b55d-823786c1a6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:19.378785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede3c41-8cec-4fd3-b55d-823786c1a6bf None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:19.379269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede3c41-8cec-4fd3-b55d-823786c1a6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:19.379742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede3c41-8cec-4fd3-b55d-823786c1a6bf None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:19.380601 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede3c41-8cec-4fd3-b55d-823786c1a6bf None admin] RBAC: Adding query filter params (type=image) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:19.381112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede3c41-8cec-4fd3-b55d-823786c1a6bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:19.382797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede3c41-8cec-4fd3-b55d-823786c1a6bf None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:19.388059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 495/992] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:19 2022] GET /identity/v3/services?type=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:19.423069 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c3dbe-2dec-4fc2-8743-955a0328d840 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:19.432122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:19.461878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db69d0f-e6b2-44e2-8102-6f598a33aa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:19.462658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db69d0f-e6b2-44e2-8102-6f598a33aa35 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:19.463195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db69d0f-e6b2-44e2-8102-6f598a33aa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:19.463470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db69d0f-e6b2-44e2-8102-6f598a33aa35 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:19.473121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db69d0f-e6b2-44e2-8102-6f598a33aa35 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:19.474947 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db69d0f-e6b2-44e2-8102-6f598a33aa35 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7db69d0f-e6b2-44e2-8102-6f598a33aa35 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:12:19.488101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:19.489161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:19.490160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:19.490160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:19.490160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:12:19.490160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:12:19.490160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:12:19.490160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:12:19.490160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:12:19.490160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:12:19.490160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 27 17:12:19.490160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:19.490160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 498/993] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:12:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:19.495266 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede3c41-8cec-4fd3-b55d-823786c1a6bf None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:19.613521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede3c41-8cec-4fd3-b55d-823786c1a6bf None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:19.649555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77aec9-1000-4118-b329-e6c6ab045b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:19.650549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77aec9-1000-4118-b329-e6c6ab045b0d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:19.651043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77aec9-1000-4118-b329-e6c6ab045b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:19.651511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77aec9-1000-4118-b329-e6c6ab045b0d None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:19.652417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77aec9-1000-4118-b329-e6c6ab045b0d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:19.652915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77aec9-1000-4118-b329-e6c6ab045b0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:19.654652 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77aec9-1000-4118-b329-e6c6ab045b0d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:19.692448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 496/994] 10.222.0.55 () {56 vars in 1101 bytes} [Tue Sep 27 17:12:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:19.942657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127e8bb-a716-4405-a7ad-9e1de6094e61 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:19.943069 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127e8bb-a716-4405-a7ad-9e1de6094e61 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:19.943345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127e8bb-a716-4405-a7ad-9e1de6094e61 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:19.947759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 499/995] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:19.953985 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8760e4c-18d2-4837-b721-b4442f76a8f6 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:19.954290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8760e4c-18d2-4837-b721-b4442f76a8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:19.954562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8760e4c-18d2-4837-b721-b4442f76a8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:19.974594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f8760e4c-18d2-4837-b721-b4442f76a8f6 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:19.975803 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f8760e4c-18d2-4837-b721-b4442f76a8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:19.986944 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:20.044989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 497/996] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:20.315363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127e8bb-a716-4405-a7ad-9e1de6094e61 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:20.418309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127e8bb-a716-4405-a7ad-9e1de6094e61 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:20.450218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30c49a-fd51-40e3-910f-977a2ebc8efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:20.450875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30c49a-fd51-40e3-910f-977a2ebc8efd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:20.451095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30c49a-fd51-40e3-910f-977a2ebc8efd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:20.451360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30c49a-fd51-40e3-910f-977a2ebc8efd None admin] PATH_INFO: `/v3/services/compute` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:20.453478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30c49a-fd51-40e3-910f-977a2ebc8efd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:20.454853 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b30c49a-fd51-40e3-910f-977a2ebc8efd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-6b30c49a-fd51-40e3-910f-977a2ebc8efd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:20.463530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:20.464504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 17:12:20.465372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:20.469285 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 500/997] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:12:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:20.471219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8760e4c-18d2-4837-b721-b4442f76a8f6 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:20.565733 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8760e4c-18d2-4837-b721-b4442f76a8f6 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:20.598587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138e620-4c5c-4890-aead-6a61a1b7bcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:20.599181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138e620-4c5c-4890-aead-6a61a1b7bcda None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:20.599452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138e620-4c5c-4890-aead-6a61a1b7bcda None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:20.599741 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138e620-4c5c-4890-aead-6a61a1b7bcda None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:20.600396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138e620-4c5c-4890-aead-6a61a1b7bcda None admin] RBAC: Adding query filter params (name=compute) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:20.600698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138e620-4c5c-4890-aead-6a61a1b7bcda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:20.602045 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138e620-4c5c-4890-aead-6a61a1b7bcda None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:20.607097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 498/998] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Sep 27 17:12:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:20.613195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30c49a-fd51-40e3-910f-977a2ebc8efd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:20.641207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:20.713918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b30c49a-fd51-40e3-910f-977a2ebc8efd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:20.748155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771af893-c2e3-4667-a4af-1aa125935b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:20.749304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771af893-c2e3-4667-a4af-1aa125935b94 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:20.749577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771af893-c2e3-4667-a4af-1aa125935b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:20.749841 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771af893-c2e3-4667-a4af-1aa125935b94 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:20.750491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771af893-c2e3-4667-a4af-1aa125935b94 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:20.750783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771af893-c2e3-4667-a4af-1aa125935b94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:20.752191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771af893-c2e3-4667-a4af-1aa125935b94 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:20.761649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 501/999] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Sep 27 17:12:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:20.762836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138e620-4c5c-4890-aead-6a61a1b7bcda None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:20.773987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:20.865947 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138e620-4c5c-4890-aead-6a61a1b7bcda None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:20.899116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc8d85d-bbfb-4b4e-a6a3-018c2788b4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:20.899715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8d85d-bbfb-4b4e-a6a3-018c2788b4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:20.899987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8d85d-bbfb-4b4e-a6a3-018c2788b4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:20.900252 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc8d85d-bbfb-4b4e-a6a3-018c2788b4d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:20.901003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8d85d-bbfb-4b4e-a6a3-018c2788b4d7 None admin] RBAC: Adding query filter params (interface=public, service_id=8008a6025d464dbd8bab9e1f9ced6dd9, region_id=RegionOne) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:20.901309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8d85d-bbfb-4b4e-a6a3-018c2788b4d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:20.902691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc8d85d-bbfb-4b4e-a6a3-018c2788b4d7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:20.912721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 499/1000] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Sep 27 17:12:20 2022] GET /identity/v3/endpoints?service_id=8008a6025d464dbd8bab9e1f9ced6dd9&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:20.913927 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771af893-c2e3-4667-a4af-1aa125935b94 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:21.005106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:21.015286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771af893-c2e3-4667-a4af-1aa125935b94 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:21.048957 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c29f73-d91d-4bad-8739-3b9f85992e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:21.049554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c29f73-d91d-4bad-8739-3b9f85992e2a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:21.049828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c29f73-d91d-4bad-8739-3b9f85992e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:21.050115 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c29f73-d91d-4bad-8739-3b9f85992e2a None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:21.050718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c29f73-d91d-4bad-8739-3b9f85992e2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:21.052059 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c29f73-d91d-4bad-8739-3b9f85992e2a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:21.060050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 502/1001] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Sep 27 17:12:20 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:12:21.259500 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efe2c1-8780-44de-91ab-2672b1755131 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:21.259705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efe2c1-8780-44de-91ab-2672b1755131 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:21.259978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efe2c1-8780-44de-91ab-2672b1755131 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:21.261455 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 500/1002] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:21.266746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ab13d-aae0-43e4-afae-4f4c8c04307d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:21.267027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ab13d-aae0-43e4-afae-4f4c8c04307d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:21.267293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ab13d-aae0-43e4-afae-4f4c8c04307d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:21.285828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-901ab13d-aae0-43e4-afae-4f4c8c04307d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:21.287365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-901ab13d-aae0-43e4-afae-4f4c8c04307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:21.360493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 503/1003] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:21.625187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efe2c1-8780-44de-91ab-2672b1755131 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:21.660707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3b83f-f964-4519-8dbe-496b53762536 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:21.661216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3b83f-f964-4519-8dbe-496b53762536 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:21.661304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3b83f-f964-4519-8dbe-496b53762536 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:21.662623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 504/1004] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:21.667504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24184481-ff7a-4c75-bc7f-a66dc35d839c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:21.667805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24184481-ff7a-4c75-bc7f-a66dc35d839c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:21.668104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24184481-ff7a-4c75-bc7f-a66dc35d839c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:21.685715 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-24184481-ff7a-4c75-bc7f-a66dc35d839c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:21.687455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-24184481-ff7a-4c75-bc7f-a66dc35d839c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:21.699340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:21.732943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70efe2c1-8780-44de-91ab-2672b1755131 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:21.764155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 505/1005] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:21.767968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e443d2-3432-4390-ad6e-53ed25baa8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:21.768602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e443d2-3432-4390-ad6e-53ed25baa8ac None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:21.768879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e443d2-3432-4390-ad6e-53ed25baa8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:21.769150 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e443d2-3432-4390-ad6e-53ed25baa8ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:21.769727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e443d2-3432-4390-ad6e-53ed25baa8ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:21.771119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e443d2-3432-4390-ad6e-53ed25baa8ac None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-22e443d2-3432-4390-ad6e-53ed25baa8ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:21.780529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:21.781533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 27 17:12:21.782412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:21.786374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 501/1006] 10.222.0.55 () {56 vars in 1126 bytes} [Tue Sep 27 17:12:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:21.787536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24184481-ff7a-4c75-bc7f-a66dc35d839c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:21.805483 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:21.891970 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24184481-ff7a-4c75-bc7f-a66dc35d839c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:21.926122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2fa05-4787-4ded-ab72-d82a35806bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:21.926725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2fa05-4787-4ded-ab72-d82a35806bea None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:21.926999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2fa05-4787-4ded-ab72-d82a35806bea None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:21.927269 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2fa05-4787-4ded-ab72-d82a35806bea None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:21.927901 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2fa05-4787-4ded-ab72-d82a35806bea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:21.928200 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2fa05-4787-4ded-ab72-d82a35806bea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:21.929578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e2fa05-4787-4ded-ab72-d82a35806bea None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:21.935183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 506/1007] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Sep 27 17:12:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:21.941421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e443d2-3432-4390-ad6e-53ed25baa8ac None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:22.033336 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2fa05-4787-4ded-ab72-d82a35806bea None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:22.041300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e443d2-3432-4390-ad6e-53ed25baa8ac None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:22.077618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee9edb-f0cd-44f4-9f94-7ca4f6cd5f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:22.078228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee9edb-f0cd-44f4-9f94-7ca4f6cd5f21 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:22.078505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee9edb-f0cd-44f4-9f94-7ca4f6cd5f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:22.078768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee9edb-f0cd-44f4-9f94-7ca4f6cd5f21 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:22.079391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee9edb-f0cd-44f4-9f94-7ca4f6cd5f21 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:22.079687 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee9edb-f0cd-44f4-9f94-7ca4f6cd5f21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:22.081044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee9edb-f0cd-44f4-9f94-7ca4f6cd5f21 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:22.083064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:22.091509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 502/1008] 10.222.0.55 () {56 vars in 1122 bytes} [Tue Sep 27 17:12:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:22.092347 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee9edb-f0cd-44f4-9f94-7ca4f6cd5f21 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:22.113533 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:22.137827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2fa05-4787-4ded-ab72-d82a35806bea None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:22.176405 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58763770-a4c0-4009-8eb2-8bab774fc708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:22.176945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58763770-a4c0-4009-8eb2-8bab774fc708 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:22.177417 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58763770-a4c0-4009-8eb2-8bab774fc708 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:22.177895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58763770-a4c0-4009-8eb2-8bab774fc708 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:22.178990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58763770-a4c0-4009-8eb2-8bab774fc708 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:22.181178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58763770-a4c0-4009-8eb2-8bab774fc708 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:22.201261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 507/1009] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:12:22 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:12:22.214818 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee9edb-f0cd-44f4-9f94-7ca4f6cd5f21 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:22.261955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575ebf7-74c5-4b96-9350-4dfa9e296978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:22.262623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575ebf7-74c5-4b96-9350-4dfa9e296978 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:22.262904 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575ebf7-74c5-4b96-9350-4dfa9e296978 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:22.263175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575ebf7-74c5-4b96-9350-4dfa9e296978 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:22.263960 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575ebf7-74c5-4b96-9350-4dfa9e296978 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:22.265967 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575ebf7-74c5-4b96-9350-4dfa9e296978 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:22.294119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 503/1010] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:12:22 2022] POST /identity/v3/endpoints => generated 325 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:23.231289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1afc78-f2b9-4c61-a914-0dd9f5fb3f01 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:23.231813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1afc78-f2b9-4c61-a914-0dd9f5fb3f01 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:23.231813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1afc78-f2b9-4c61-a914-0dd9f5fb3f01 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:23.233350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 508/1011] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:23.238297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bf58d-7acb-40d1-8954-4d61df49b4af None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:23.238686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bf58d-7acb-40d1-8954-4d61df49b4af None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:23.238842 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bf58d-7acb-40d1-8954-4d61df49b4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:23.256939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-139bf58d-7acb-40d1-8954-4d61df49b4af None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:23.258138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-139bf58d-7acb-40d1-8954-4d61df49b4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:23.319658 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:23.341555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 504/1012] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:23 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:23.610358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1afc78-f2b9-4c61-a914-0dd9f5fb3f01 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:23.711741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1afc78-f2b9-4c61-a914-0dd9f5fb3f01 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:23.744493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50afe2ce-9fd3-4964-99ce-48e83fa94d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:23.745103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afe2ce-9fd3-4964-99ce-48e83fa94d19 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:23.745379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afe2ce-9fd3-4964-99ce-48e83fa94d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:23.745648 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afe2ce-9fd3-4964-99ce-48e83fa94d19 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:23.746247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50afe2ce-9fd3-4964-99ce-48e83fa94d19 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:23.747591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50afe2ce-9fd3-4964-99ce-48e83fa94d19 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-50afe2ce-9fd3-4964-99ce-48e83fa94d19 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:23.756132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:23.757091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 27 17:12:23.757940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:23.761979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 509/1013] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:12:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:23.763274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bf58d-7acb-40d1-8954-4d61df49b4af None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:23.861953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bf58d-7acb-40d1-8954-4d61df49b4af None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:23.902122 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07400472-504b-432a-bf5e-5e52fc4750d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:23.902734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07400472-504b-432a-bf5e-5e52fc4750d1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:23.903005 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07400472-504b-432a-bf5e-5e52fc4750d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:23.903281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07400472-504b-432a-bf5e-5e52fc4750d1 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:23.903900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07400472-504b-432a-bf5e-5e52fc4750d1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:23.904196 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07400472-504b-432a-bf5e-5e52fc4750d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:23.906304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07400472-504b-432a-bf5e-5e52fc4750d1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:23.916413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 505/1014] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Sep 27 17:12:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:23.918134 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50afe2ce-9fd3-4964-99ce-48e83fa94d19 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:23.922407 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:24.025530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50afe2ce-9fd3-4964-99ce-48e83fa94d19 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:24.059939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178843e6-045f-4514-b102-84e456babd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:24.060536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178843e6-045f-4514-b102-84e456babd0c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:24.060809 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178843e6-045f-4514-b102-84e456babd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:24.061124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178843e6-045f-4514-b102-84e456babd0c None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:24.061749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178843e6-045f-4514-b102-84e456babd0c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:24.062044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178843e6-045f-4514-b102-84e456babd0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:24.063420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178843e6-045f-4514-b102-84e456babd0c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:24.072790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 510/1015] 10.222.0.55 () {56 vars in 1112 bytes} [Tue Sep 27 17:12:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:24.074082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07400472-504b-432a-bf5e-5e52fc4750d1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:24.122685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:24.176696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07400472-504b-432a-bf5e-5e52fc4750d1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:24.212363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe836de-fb5b-4167-abe8-d513058c6c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:24.213361 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe836de-fb5b-4167-abe8-d513058c6c24 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:24.213835 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe836de-fb5b-4167-abe8-d513058c6c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:24.214341 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe836de-fb5b-4167-abe8-d513058c6c24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:24.215543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe836de-fb5b-4167-abe8-d513058c6c24 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:24.217162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe836de-fb5b-4167-abe8-d513058c6c24 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:24.240984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 506/1016] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:12:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:24.421186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccf71a-6116-474c-afde-6c89b5482d2a None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:24.421696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccf71a-6116-474c-afde-6c89b5482d2a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:24.421696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ccf71a-6116-474c-afde-6c89b5482d2a None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:24.423542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 511/1017] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:24.428536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d08b60-c380-4e93-9cab-de19bd01db29 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:24.428822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d08b60-c380-4e93-9cab-de19bd01db29 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:24.429095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d08b60-c380-4e93-9cab-de19bd01db29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:24.448797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-34d08b60-c380-4e93-9cab-de19bd01db29 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:24.449974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-34d08b60-c380-4e93-9cab-de19bd01db29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:24.490160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c0bc7-58b6-4b80-a6f2-43dfd19c7245 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:24.490446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c0bc7-58b6-4b80-a6f2-43dfd19c7245 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:24.490723 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c0bc7-58b6-4b80-a6f2-43dfd19c7245 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:24.492054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 512/1018] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:24.497738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996daf48-88ca-45d1-8da8-aea601f4b916 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:24.498017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996daf48-88ca-45d1-8da8-aea601f4b916 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:24.498326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996daf48-88ca-45d1-8da8-aea601f4b916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:24.518038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-996daf48-88ca-45d1-8da8-aea601f4b916 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:24.519378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-996daf48-88ca-45d1-8da8-aea601f4b916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:24.532912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 507/1019] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:24.592181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:24.602116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 513/1020] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:24.802300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d08b60-c380-4e93-9cab-de19bd01db29 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:24.831049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:24.872628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996daf48-88ca-45d1-8da8-aea601f4b916 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:24.910885 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d08b60-c380-4e93-9cab-de19bd01db29 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:24.951932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4000f0-a122-4fc4-bb4d-3dd29e236aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:24.952669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4000f0-a122-4fc4-bb4d-3dd29e236aed None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:24.952948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4000f0-a122-4fc4-bb4d-3dd29e236aed None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:24.953214 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4000f0-a122-4fc4-bb4d-3dd29e236aed None admin] PATH_INFO: `/v3/services/image` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:24.953794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4000f0-a122-4fc4-bb4d-3dd29e236aed None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:24.955283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4000f0-a122-4fc4-bb4d-3dd29e236aed None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-eb4000f0-a122-4fc4-bb4d-3dd29e236aed None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:24.963931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:24.964959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:24.965821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 508/1021] 10.222.0.55 () {56 vars in 1105 bytes} [Tue Sep 27 17:12:24 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:24.970836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4000f0-a122-4fc4-bb4d-3dd29e236aed None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:24.996616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996daf48-88ca-45d1-8da8-aea601f4b916 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:25.036471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44340e0-ae79-4cc8-9c1c-958f145b1ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:25.037095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44340e0-ae79-4cc8-9c1c-958f145b1ece None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:25.037369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44340e0-ae79-4cc8-9c1c-958f145b1ece None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:25.037635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44340e0-ae79-4cc8-9c1c-958f145b1ece None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:25.038729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44340e0-ae79-4cc8-9c1c-958f145b1ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:25.040139 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44340e0-ae79-4cc8-9c1c-958f145b1ece None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:25.041725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 514/1022] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:25.048710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44340e0-ae79-4cc8-9c1c-958f145b1ece None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:25.070850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:25.081546 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4000f0-a122-4fc4-bb4d-3dd29e236aed None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:25.115488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:25.119038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810575ba-1280-4438-afb5-4bfc7a9b2bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:25.119646 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810575ba-1280-4438-afb5-4bfc7a9b2bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:25.119925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810575ba-1280-4438-afb5-4bfc7a9b2bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:25.120185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810575ba-1280-4438-afb5-4bfc7a9b2bd7 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:25.120815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810575ba-1280-4438-afb5-4bfc7a9b2bd7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:25.121112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810575ba-1280-4438-afb5-4bfc7a9b2bd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:25.122504 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810575ba-1280-4438-afb5-4bfc7a9b2bd7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:25.127969 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 509/1023] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:25.133572 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810575ba-1280-4438-afb5-4bfc7a9b2bd7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:25.158764 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44340e0-ae79-4cc8-9c1c-958f145b1ece None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:25.193993 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d9a27-5b2d-40ca-8917-995993447b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:25.194616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d9a27-5b2d-40ca-8917-995993447b42 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:25.194887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d9a27-5b2d-40ca-8917-995993447b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:25.195150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d9a27-5b2d-40ca-8917-995993447b42 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:25.195782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d9a27-5b2d-40ca-8917-995993447b42 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:25.197178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d9a27-5b2d-40ca-8917-995993447b42 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:25.224893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 515/1024] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:12:25 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:25.252385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810575ba-1280-4438-afb5-4bfc7a9b2bd7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:25.293605 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4225faa-5c39-4536-836f-755b99d28a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:25.294685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4225faa-5c39-4536-836f-755b99d28a06 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:25.295164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4225faa-5c39-4536-836f-755b99d28a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:25.296279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4225faa-5c39-4536-836f-755b99d28a06 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:25.297125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4225faa-5c39-4536-836f-755b99d28a06 None admin] RBAC: Adding query filter params (type=image) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:25.297636 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4225faa-5c39-4536-836f-755b99d28a06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:25.299301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4225faa-5c39-4536-836f-755b99d28a06 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:25.313204 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 510/1025] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:25.315362 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d9a27-5b2d-40ca-8917-995993447b42 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:25.428678 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d9a27-5b2d-40ca-8917-995993447b42 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:25.444553 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:25.464705 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5153e-c47f-4337-88d8-4d6a0b92d168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:25.465309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5153e-c47f-4337-88d8-4d6a0b92d168 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:25.465584 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5153e-c47f-4337-88d8-4d6a0b92d168 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:25.465853 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5153e-c47f-4337-88d8-4d6a0b92d168 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:25.466583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5153e-c47f-4337-88d8-4d6a0b92d168 None admin] RBAC: Adding query filter params (interface=public, service_id=61583c99fd664589ab7727df92c2b94b, region_id=RegionOne) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:25.466892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5153e-c47f-4337-88d8-4d6a0b92d168 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:25.468851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5153e-c47f-4337-88d8-4d6a0b92d168 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:25.474357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 516/1026] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Sep 27 17:12:25 2022] GET /identity/v3/endpoints?service_id=61583c99fd664589ab7727df92c2b94b&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:25.480027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4225faa-5c39-4536-836f-755b99d28a06 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:25.503553 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:25.582817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4225faa-5c39-4536-836f-755b99d28a06 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:25.616958 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910e757-7914-4c73-9b59-cf0c1282d033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:25.617949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910e757-7914-4c73-9b59-cf0c1282d033 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:25.618400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910e757-7914-4c73-9b59-cf0c1282d033 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:25.618850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910e757-7914-4c73-9b59-cf0c1282d033 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:25.619590 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910e757-7914-4c73-9b59-cf0c1282d033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:25.621081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910e757-7914-4c73-9b59-cf0c1282d033 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:25.628381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 511/1027] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Sep 27 17:12:25 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:12:26.547724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61949886-aae8-40e4-9e29-95d7ba806c89 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:26.548241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61949886-aae8-40e4-9e29-95d7ba806c89 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:26.548241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61949886-aae8-40e4-9e29-95d7ba806c89 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:26.549772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 517/1028] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:26.554762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd3a8-174d-49ff-bbe9-5f1110acb9ae None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:26.555144 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd3a8-174d-49ff-bbe9-5f1110acb9ae None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:26.555319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd3a8-174d-49ff-bbe9-5f1110acb9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:26.573290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-523dd3a8-174d-49ff-bbe9-5f1110acb9ae None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:26.574440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-523dd3a8-174d-49ff-bbe9-5f1110acb9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:26.642194 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 512/1029] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:26.908635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61949886-aae8-40e4-9e29-95d7ba806c89 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:27.010637 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61949886-aae8-40e4-9e29-95d7ba806c89 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:27.043214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd424fac-f26f-4416-88e4-b220c4a84d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:27.043832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd424fac-f26f-4416-88e4-b220c4a84d8a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:27.044107 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd424fac-f26f-4416-88e4-b220c4a84d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:27.044434 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd424fac-f26f-4416-88e4-b220c4a84d8a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:27.054926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd424fac-f26f-4416-88e4-b220c4a84d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-fd424fac-f26f-4416-88e4-b220c4a84d8a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:12:27.059680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:27.060606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:27.062044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:27.062044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:27.062044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:27.062044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:27.062044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:12:27.062044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:12:27.062044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:12:27.062044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:12:27.062044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:12:27.062044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:12:27.062044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 17:12:27.062044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:27.066395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 518/1030] 10.222.0.55 () {56 vars in 1120 bytes} [Tue Sep 27 17:12:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:27.067217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523dd3a8-174d-49ff-bbe9-5f1110acb9ae None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:27.082574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:27.168010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523dd3a8-174d-49ff-bbe9-5f1110acb9ae None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:27.201912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240cbfe-b82b-4818-b4fc-fcd8d83968c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:27.202543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240cbfe-b82b-4818-b4fc-fcd8d83968c0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:27.202819 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240cbfe-b82b-4818-b4fc-fcd8d83968c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:27.203087 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f240cbfe-b82b-4818-b4fc-fcd8d83968c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:27.203748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240cbfe-b82b-4818-b4fc-fcd8d83968c0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:27.204045 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240cbfe-b82b-4818-b4fc-fcd8d83968c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:27.205402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f240cbfe-b82b-4818-b4fc-fcd8d83968c0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:27.213436 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 513/1031] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Sep 27 17:12:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:27.219052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd424fac-f26f-4416-88e4-b220c4a84d8a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:27.260885 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:27.330659 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd424fac-f26f-4416-88e4-b220c4a84d8a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:27.365756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6e4ed-386f-4147-b549-a6045ab350a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:27.366406 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6e4ed-386f-4147-b549-a6045ab350a1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:27.366682 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6e4ed-386f-4147-b549-a6045ab350a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:27.366951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6e4ed-386f-4147-b549-a6045ab350a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:27.378025 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6e4ed-386f-4147-b549-a6045ab350a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:27.379454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6e4ed-386f-4147-b549-a6045ab350a1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7cf6e4ed-386f-4147-b549-a6045ab350a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:27.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:27.394660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:27.395875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:27.395875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:27.395875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:27.398421 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 519/1032] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:27.400237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240cbfe-b82b-4818-b4fc-fcd8d83968c0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:27.429184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09979a97-8985-4464-b60f-c98d34168e53 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:27.429360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09979a97-8985-4464-b60f-c98d34168e53 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:27.429624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09979a97-8985-4464-b60f-c98d34168e53 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:27.431238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 520/1033] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:27.436920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c713bf-edc3-4ad6-82d8-747846b9d5c4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:27.437197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c713bf-edc3-4ad6-82d8-747846b9d5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:27.437466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c713bf-edc3-4ad6-82d8-747846b9d5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:27.456500 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c3c713bf-edc3-4ad6-82d8-747846b9d5c4 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:27.458032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c3c713bf-edc3-4ad6-82d8-747846b9d5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:27.486537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:27.511105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f240cbfe-b82b-4818-b4fc-fcd8d83968c0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:27.530858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 521/1034] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:27.548810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003093d-f809-4780-b7fe-22ed488eb153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:27.549507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003093d-f809-4780-b7fe-22ed488eb153 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:27.549850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003093d-f809-4780-b7fe-22ed488eb153 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:27.550209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003093d-f809-4780-b7fe-22ed488eb153 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:27.550912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003093d-f809-4780-b7fe-22ed488eb153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:27.551274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003093d-f809-4780-b7fe-22ed488eb153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:27.552689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003093d-f809-4780-b7fe-22ed488eb153 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:27.569543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 514/1035] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:27.571190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c713bf-edc3-4ad6-82d8-747846b9d5c4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:27.595575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:27.671571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c713bf-edc3-4ad6-82d8-747846b9d5c4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:27.708925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a89fda-b608-4947-9a3e-d7d1b6f89554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:27.709473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a89fda-b608-4947-9a3e-d7d1b6f89554 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:27.709748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a89fda-b608-4947-9a3e-d7d1b6f89554 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:27.710016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a89fda-b608-4947-9a3e-d7d1b6f89554 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:27.717292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a89fda-b608-4947-9a3e-d7d1b6f89554 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:27.718748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a89fda-b608-4947-9a3e-d7d1b6f89554 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-81a89fda-b608-4947-9a3e-d7d1b6f89554 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:12:27.730250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:27.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:27.732120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:27.732120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:27.732120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:12:27.732120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:12:27.732120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:12:27.732120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:12:27.732120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:12:27.732120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:12:27.732120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:12:27.732120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:27.734410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 522/1036] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Sep 27 17:12:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:27.737144 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003093d-f809-4780-b7fe-22ed488eb153 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:27.793554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e1e64-a090-4132-891c-13c4b4d6c3df None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:27.793876 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e1e64-a090-4132-891c-13c4b4d6c3df None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:27.794134 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e1e64-a090-4132-891c-13c4b4d6c3df None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:27.795665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 523/1037] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:27.798969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e1e64-a090-4132-891c-13c4b4d6c3df None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:27.838062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003093d-f809-4780-b7fe-22ed488eb153 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:27.870641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4650329-b11c-4612-9851-ca947367f5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:27.871236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4650329-b11c-4612-9851-ca947367f5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:27.871510 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4650329-b11c-4612-9851-ca947367f5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:27.871776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4650329-b11c-4612-9851-ca947367f5a9 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:27.872495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4650329-b11c-4612-9851-ca947367f5a9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:27.872804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4650329-b11c-4612-9851-ca947367f5a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:27.874799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4650329-b11c-4612-9851-ca947367f5a9 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:27.877265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:27.880571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:27.893293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 515/1038] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Sep 27 17:12:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:27.894527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb2e1d-12ed-46c8-8a14-0be170afd365 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:27.894818 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb2e1d-12ed-46c8-8a14-0be170afd365 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:27.895146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb2e1d-12ed-46c8-8a14-0be170afd365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:27.901183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e1e64-a090-4132-891c-13c4b4d6c3df None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:27.912151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-eceb2e1d-12ed-46c8-8a14-0be170afd365 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:27.913335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-eceb2e1d-12ed-46c8-8a14-0be170afd365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:27.935377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79d428-eaac-4a78-b188-c812faa95754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:27.936553 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79d428-eaac-4a78-b188-c812faa95754 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:27.936835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79d428-eaac-4a78-b188-c812faa95754 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:27.937102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79d428-eaac-4a78-b188-c812faa95754 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:27.938229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79d428-eaac-4a78-b188-c812faa95754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:27.939677 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79d428-eaac-4a78-b188-c812faa95754 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:27.941255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 524/1039] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:27.942010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79d428-eaac-4a78-b188-c812faa95754 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:27.985807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 516/1040] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:27.986583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb2e1d-12ed-46c8-8a14-0be170afd365 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:28.050151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79d428-eaac-4a78-b188-c812faa95754 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:28.085257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc3e5d-a12a-41f9-b154-20b0c8119a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:28.085854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc3e5d-a12a-41f9-b154-20b0c8119a67 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:28.086152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc3e5d-a12a-41f9-b154-20b0c8119a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:28.086421 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc3e5d-a12a-41f9-b154-20b0c8119a67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:28.087386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb2e1d-12ed-46c8-8a14-0be170afd365 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:28.106267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:28.114278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:28.126166 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8887c4a-456b-4e9d-970a-a722a2a33d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:28.127011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8887c4a-456b-4e9d-970a-a722a2a33d0a None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:28.127451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8887c4a-456b-4e9d-970a-a722a2a33d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:28.127877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8887c4a-456b-4e9d-970a-a722a2a33d0a None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:28.128522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8887c4a-456b-4e9d-970a-a722a2a33d0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:28.129951 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8887c4a-456b-4e9d-970a-a722a2a33d0a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:28.158104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 517/1041] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:12:27 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:28.177368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc3e5d-a12a-41f9-b154-20b0c8119a67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:28.178900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc3e5d-a12a-41f9-b154-20b0c8119a67 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:28.258159 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8887c4a-456b-4e9d-970a-a722a2a33d0a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:28.290148 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 525/1042] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:12:27 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:12:28.296732 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc3e5d-a12a-41f9-b154-20b0c8119a67 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:28.360787 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:28.389509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8887c4a-456b-4e9d-970a-a722a2a33d0a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:28.420694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc3e5d-a12a-41f9-b154-20b0c8119a67 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:28.427846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa20fdf-8e51-477f-a53c-5c032871a326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:28.428485 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa20fdf-8e51-477f-a53c-5c032871a326 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:28.428753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa20fdf-8e51-477f-a53c-5c032871a326 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:28.429027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa20fdf-8e51-477f-a53c-5c032871a326 None admin] PATH_INFO: `/v3/services/image` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:28.429616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa20fdf-8e51-477f-a53c-5c032871a326 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:28.431016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa20fdf-8e51-477f-a53c-5c032871a326 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-4aa20fdf-8e51-477f-a53c-5c032871a326 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:28.440399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:28.441435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:28.442790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 518/1043] 10.222.0.55 () {56 vars in 1105 bytes} [Tue Sep 27 17:12:28 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:28.447275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa20fdf-8e51-477f-a53c-5c032871a326 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:28.464698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea4803-5ffb-4d21-9fc7-987bcfcd0164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:28.465610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea4803-5ffb-4d21-9fc7-987bcfcd0164 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:28.465939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea4803-5ffb-4d21-9fc7-987bcfcd0164 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:28.466247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea4803-5ffb-4d21-9fc7-987bcfcd0164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:28.480450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea4803-5ffb-4d21-9fc7-987bcfcd0164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:28.481882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea4803-5ffb-4d21-9fc7-987bcfcd0164 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:28.494307 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-36ea4803-5ffb-4d21-9fc7-987bcfcd0164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:28.499685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:28.500649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:28.501572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:28.501572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:28.501572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:28.501572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 526/1044] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:28.506561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea4803-5ffb-4d21-9fc7-987bcfcd0164 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:28.559732 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa20fdf-8e51-477f-a53c-5c032871a326 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:28.598368 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fe08d-e66a-4449-ba1a-8a63cff9a996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:28.599876 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fe08d-e66a-4449-ba1a-8a63cff9a996 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:28.600450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fe08d-e66a-4449-ba1a-8a63cff9a996 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:28.600938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fe08d-e66a-4449-ba1a-8a63cff9a996 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:28.601737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fe08d-e66a-4449-ba1a-8a63cff9a996 None admin] RBAC: Adding query filter params (name=image) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:28.602319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fe08d-e66a-4449-ba1a-8a63cff9a996 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:28.604020 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fe08d-e66a-4449-ba1a-8a63cff9a996 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:28.608876 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea4803-5ffb-4d21-9fc7-987bcfcd0164 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:28.610404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:28.610704 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 519/1045] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:28.616254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fe08d-e66a-4449-ba1a-8a63cff9a996 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:28.649728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed30c1-612f-43d8-81e4-94f85ad16200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:28.650356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed30c1-612f-43d8-81e4-94f85ad16200 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:28.650632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed30c1-612f-43d8-81e4-94f85ad16200 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:28.650901 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed30c1-612f-43d8-81e4-94f85ad16200 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:28.651679 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed30c1-612f-43d8-81e4-94f85ad16200 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:28.651982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed30c1-612f-43d8-81e4-94f85ad16200 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:28.653335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed30c1-612f-43d8-81e4-94f85ad16200 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:28.665997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 527/1046] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:28.671943 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed30c1-612f-43d8-81e4-94f85ad16200 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:28.718640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fe08d-e66a-4449-ba1a-8a63cff9a996 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:28.727536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:28.755960 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86909341-1e64-436d-bc14-8a3be991e355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:28.756570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86909341-1e64-436d-bc14-8a3be991e355 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:28.756963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86909341-1e64-436d-bc14-8a3be991e355 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:28.757237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86909341-1e64-436d-bc14-8a3be991e355 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:28.757892 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86909341-1e64-436d-bc14-8a3be991e355 None admin] RBAC: Adding query filter params (type=image) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:28.758217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86909341-1e64-436d-bc14-8a3be991e355 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:28.759708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86909341-1e64-436d-bc14-8a3be991e355 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:28.765714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 520/1047] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:28.771248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86909341-1e64-436d-bc14-8a3be991e355 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:28.780081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed30c1-612f-43d8-81e4-94f85ad16200 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:28.821490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88281222-327d-4088-af47-3d858c5fc8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:28.822034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88281222-327d-4088-af47-3d858c5fc8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:28.822336 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88281222-327d-4088-af47-3d858c5fc8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:28.822606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88281222-327d-4088-af47-3d858c5fc8d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:28.829970 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88281222-327d-4088-af47-3d858c5fc8d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:28.831743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88281222-327d-4088-af47-3d858c5fc8d2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-88281222-327d-4088-af47-3d858c5fc8d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:12:28.842931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:28.843865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:28.844772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:12:28.844772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:12:28.844772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:12:28.844772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:12:28.844772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:12:28.844772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:12:28.844772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:28.844772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:28.849183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 528/1048] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:12:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:28.849677 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88281222-327d-4088-af47-3d858c5fc8d2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:28.874830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86909341-1e64-436d-bc14-8a3be991e355 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:28.875813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:28.915399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187754a-5d3c-463c-9d6b-872238d7ead7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:28.916015 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187754a-5d3c-463c-9d6b-872238d7ead7 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:28.916297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187754a-5d3c-463c-9d6b-872238d7ead7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:28.916565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187754a-5d3c-463c-9d6b-872238d7ead7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:28.917233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187754a-5d3c-463c-9d6b-872238d7ead7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:28.918640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187754a-5d3c-463c-9d6b-872238d7ead7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:28.940694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 521/1049] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:12:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:28.976734 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88281222-327d-4088-af47-3d858c5fc8d2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:29.023109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e10849-ff89-45ce-b2e2-246c1234358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:29.024010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e10849-ff89-45ce-b2e2-246c1234358b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:29.024441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e10849-ff89-45ce-b2e2-246c1234358b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:29.024856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e10849-ff89-45ce-b2e2-246c1234358b None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:29.025754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e10849-ff89-45ce-b2e2-246c1234358b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:29.026399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e10849-ff89-45ce-b2e2-246c1234358b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:29.028145 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e10849-ff89-45ce-b2e2-246c1234358b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:29.043066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 529/1050] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:12:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:29.049901 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187754a-5d3c-463c-9d6b-872238d7ead7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:29.083367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:29.164475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187754a-5d3c-463c-9d6b-872238d7ead7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:29.198985 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8086a5-70b6-4f37-b7d6-97b7cbea6128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:29.199590 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8086a5-70b6-4f37-b7d6-97b7cbea6128 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:29.199918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8086a5-70b6-4f37-b7d6-97b7cbea6128 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:29.200123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8086a5-70b6-4f37-b7d6-97b7cbea6128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:29.200853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8086a5-70b6-4f37-b7d6-97b7cbea6128 None admin] RBAC: Adding query filter params (role.id=a2cb336de6ca492ab46d7dbad02e31a0, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=5537185f42474cfd86fca70fa8beddd2) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:29.201155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8086a5-70b6-4f37-b7d6-97b7cbea6128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:29.202652 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8086a5-70b6-4f37-b7d6-97b7cbea6128 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:29.208621 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 522/1051] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:12:29 2022] GET /identity/v3/role_assignments?user.id=5537185f42474cfd86fca70fa8beddd2&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=a2cb336de6ca492ab46d7dbad02e31a0 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:30.472752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136e62d-fe29-4440-83b6-22e105e7e72f None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:30.473265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136e62d-fe29-4440-83b6-22e105e7e72f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:30.473342 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136e62d-fe29-4440-83b6-22e105e7e72f None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:30.474850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 530/1052] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:30.480046 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791f287-0103-42f7-a6bb-2abeeec60c53 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:30.480438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791f287-0103-42f7-a6bb-2abeeec60c53 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:30.480612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791f287-0103-42f7-a6bb-2abeeec60c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:30.499652 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8791f287-0103-42f7-a6bb-2abeeec60c53 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:30.500837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8791f287-0103-42f7-a6bb-2abeeec60c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:30.568131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 523/1053] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:30.840514 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136e62d-fe29-4440-83b6-22e105e7e72f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:30.927470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:30.952372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136e62d-fe29-4440-83b6-22e105e7e72f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:30.986955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34e684-8f62-4e63-aa80-7734b6b79e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:30.987585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34e684-8f62-4e63-aa80-7734b6b79e4a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:30.987860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34e684-8f62-4e63-aa80-7734b6b79e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:30.988124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34e684-8f62-4e63-aa80-7734b6b79e4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:30.998249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34e684-8f62-4e63-aa80-7734b6b79e4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0b34e684-8f62-4e63-aa80-7734b6b79e4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:12:31.002929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:31.003866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:31.004725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:31.004725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:31.004725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:31.004725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:31.004725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:12:31.004725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:12:31.004725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:12:31.004725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:12:31.004725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:12:31.004725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:12:31.004725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:12:31.004725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:31.004725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 531/1054] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:12:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:31.010399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791f287-0103-42f7-a6bb-2abeeec60c53 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:31.035335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:31.114057 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791f287-0103-42f7-a6bb-2abeeec60c53 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:31.152849 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e123e-2449-40d7-9d1f-0fd732dc6a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:31.153462 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e123e-2449-40d7-9d1f-0fd732dc6a6f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:31.153738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e123e-2449-40d7-9d1f-0fd732dc6a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:31.154008 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e123e-2449-40d7-9d1f-0fd732dc6a6f None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:31.155097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e123e-2449-40d7-9d1f-0fd732dc6a6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:31.155395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e123e-2449-40d7-9d1f-0fd732dc6a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:31.156758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e123e-2449-40d7-9d1f-0fd732dc6a6f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:31.169745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 524/1055] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:12:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:31.171759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34e684-8f62-4e63-aa80-7734b6b79e4a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:31.250417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a38cc-1535-442c-84fd-5e895b19bb9a None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:31.250767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a38cc-1535-442c-84fd-5e895b19bb9a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:31.250964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a38cc-1535-442c-84fd-5e895b19bb9a None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:31.252587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 525/1056] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:31.258496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564a599-9e15-4e58-a687-764d3b85ca70 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:31.258782 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564a599-9e15-4e58-a687-764d3b85ca70 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:31.259054 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564a599-9e15-4e58-a687-764d3b85ca70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:31.279043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2564a599-9e15-4e58-a687-764d3b85ca70 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:31.281659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2564a599-9e15-4e58-a687-764d3b85ca70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:31.290694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34e684-8f62-4e63-aa80-7734b6b79e4a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:31.331892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de1b84-9f90-4c8c-a7a2-bec5d80fcc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:31.332965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de1b84-9f90-4c8c-a7a2-bec5d80fcc4b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:31.333241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de1b84-9f90-4c8c-a7a2-bec5d80fcc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:31.333910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de1b84-9f90-4c8c-a7a2-bec5d80fcc4b None admin] PATH_INFO: `/v3/users/e55cf7c6da094849945a1b43ae15d533` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:31.344140 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:31.353064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de1b84-9f90-4c8c-a7a2-bec5d80fcc4b None admin] RBAC: Authorizing `identity:get_user(user_id=e55cf7c6da094849945a1b43ae15d533)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:31.355017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de1b84-9f90-4c8c-a7a2-bec5d80fcc4b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:31.358116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 532/1057] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:12:31 2022] GET /identity/v3/users/e55cf7c6da094849945a1b43ae15d533 => generated 285 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:31.366371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 526/1058] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:31.366515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de1b84-9f90-4c8c-a7a2-bec5d80fcc4b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:31.377052 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f908444-65c1-48e2-a4c1-1518aae53a69 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:31.377343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f908444-65c1-48e2-a4c1-1518aae53a69 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:31.377619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f908444-65c1-48e2-a4c1-1518aae53a69 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:31.378991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 527/1059] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:31.383806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb3308-5ba9-4afd-980b-0947699fb307 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:31.384087 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb3308-5ba9-4afd-980b-0947699fb307 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:31.384357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb3308-5ba9-4afd-980b-0947699fb307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:31.401970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1ebb3308-5ba9-4afd-980b-0947699fb307 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:31.403273 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1ebb3308-5ba9-4afd-980b-0947699fb307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:31.413541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:31.463037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de1b84-9f90-4c8c-a7a2-bec5d80fcc4b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:31.478123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 528/1060] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:31.499531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced34fba-0236-4349-bf7b-cdb8da7d72c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:31.500130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced34fba-0236-4349-bf7b-cdb8da7d72c3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:31.500403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced34fba-0236-4349-bf7b-cdb8da7d72c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:31.500670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced34fba-0236-4349-bf7b-cdb8da7d72c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:31.560035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:31.566872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced34fba-0236-4349-bf7b-cdb8da7d72c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:31.568778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced34fba-0236-4349-bf7b-cdb8da7d72c3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:31.630798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb3308-5ba9-4afd-980b-0947699fb307 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:31.675924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 533/1061] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:12:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 27 17:12:31.681931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced34fba-0236-4349-bf7b-cdb8da7d72c3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:31.735657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb3308-5ba9-4afd-980b-0947699fb307 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:31.770705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0932970-0ecb-43a3-992f-f5bb486af434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:31.771310 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0932970-0ecb-43a3-992f-f5bb486af434 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:31.771586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0932970-0ecb-43a3-992f-f5bb486af434 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:31.771856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0932970-0ecb-43a3-992f-f5bb486af434 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:31.777445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:31.784854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced34fba-0236-4349-bf7b-cdb8da7d72c3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:31.787664 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:31.818946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af98730-b996-4a18-bfde-8ef8772e29ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:31.819559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af98730-b996-4a18-bfde-8ef8772e29ec None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:31.819835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af98730-b996-4a18-bfde-8ef8772e29ec None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:31.820104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af98730-b996-4a18-bfde-8ef8772e29ec None admin] PATH_INFO: `/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:31.821213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af98730-b996-4a18-bfde-8ef8772e29ec None admin] RBAC: Authorizing `identity:get_project(project_id=2ec9f3d4fe3547faa80e681ff40e967c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:31.822664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af98730-b996-4a18-bfde-8ef8772e29ec None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:31.824459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 534/1062] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:12:31 2022] GET /identity/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:31.825048 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af98730-b996-4a18-bfde-8ef8772e29ec None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:31.839411 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0932970-0ecb-43a3-992f-f5bb486af434 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:31.840854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0932970-0ecb-43a3-992f-f5bb486af434 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:31.925718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af98730-b996-4a18-bfde-8ef8772e29ec None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:31.937309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 529/1063] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:12:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 27 17:12:31.938170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0932970-0ecb-43a3-992f-f5bb486af434 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:31.960363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92273f60-e7ed-465e-91ba-47a9870d948c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:31.960974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92273f60-e7ed-465e-91ba-47a9870d948c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:31.961247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92273f60-e7ed-465e-91ba-47a9870d948c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:31.961513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92273f60-e7ed-465e-91ba-47a9870d948c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:31.972292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92273f60-e7ed-465e-91ba-47a9870d948c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-92273f60-e7ed-465e-91ba-47a9870d948c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:12:31.977047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:31.978009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:31.978924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:31.978924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:31.978924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:31.978924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:31.978924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:12:31.978924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:12:31.978924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:12:31.978924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:12:31.978924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:12:31.978924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:12:31.978924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 17:12:31.978924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:31.978924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 535/1064] 10.222.0.55 () {56 vars in 1120 bytes} [Tue Sep 27 17:12:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:31.979937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92273f60-e7ed-465e-91ba-47a9870d948c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:31.983750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:32.010512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:32.042676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0932970-0ecb-43a3-992f-f5bb486af434 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:32.075969 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7458b90-c1af-4a66-9e25-42349ccf26af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:32.076595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7458b90-c1af-4a66-9e25-42349ccf26af None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:32.076868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7458b90-c1af-4a66-9e25-42349ccf26af None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:32.077142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7458b90-c1af-4a66-9e25-42349ccf26af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:32.077861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7458b90-c1af-4a66-9e25-42349ccf26af None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=2ec9f3d4fe3547faa80e681ff40e967c, user.id=e55cf7c6da094849945a1b43ae15d533) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:32.078197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7458b90-c1af-4a66-9e25-42349ccf26af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:32.079616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7458b90-c1af-4a66-9e25-42349ccf26af None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:32.084746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92273f60-e7ed-465e-91ba-47a9870d948c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:32.085235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 530/1065] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:12:31 2022] GET /identity/v3/role_assignments?user.id=e55cf7c6da094849945a1b43ae15d533&scope.project.id=2ec9f3d4fe3547faa80e681ff40e967c&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:32.085957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7458b90-c1af-4a66-9e25-42349ccf26af None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:32.125160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a071369-761c-4884-b21f-0422bba167ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:32.125807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a071369-761c-4884-b21f-0422bba167ef None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:32.126106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a071369-761c-4884-b21f-0422bba167ef None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:32.126379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a071369-761c-4884-b21f-0422bba167ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:32.138561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a071369-761c-4884-b21f-0422bba167ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:32.140163 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a071369-761c-4884-b21f-0422bba167ef None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0a071369-761c-4884-b21f-0422bba167ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:32.156879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:32.157923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:32.158960 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:32.158960 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:32.158960 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:32.158960 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 536/1066] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:32.164461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a071369-761c-4884-b21f-0422bba167ef None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:32.208770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7458b90-c1af-4a66-9e25-42349ccf26af None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:32.219105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:32.246831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf0879-a5a0-42d6-b212-668c865003a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:32.247449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf0879-a5a0-42d6-b212-668c865003a9 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:32.247728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf0879-a5a0-42d6-b212-668c865003a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:32.248526 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf0879-a5a0-42d6-b212-668c865003a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:32.249236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf0879-a5a0-42d6-b212-668c865003a9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:32.249540 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf0879-a5a0-42d6-b212-668c865003a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:32.250585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:32.250941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cf0879-a5a0-42d6-b212-668c865003a9 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:32.261001 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 531/1067] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Sep 27 17:12:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:32.267000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf0879-a5a0-42d6-b212-668c865003a9 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:32.278467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a071369-761c-4884-b21f-0422bba167ef None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:32.322048 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefa631-9bef-4256-a01d-4d8f8b1c619f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:32.322948 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefa631-9bef-4256-a01d-4d8f8b1c619f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:32.323433 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefa631-9bef-4256-a01d-4d8f8b1c619f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:32.323879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefa631-9bef-4256-a01d-4d8f8b1c619f None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:32.325013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefa631-9bef-4256-a01d-4d8f8b1c619f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:32.325398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefa631-9bef-4256-a01d-4d8f8b1c619f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:32.327685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfefa631-9bef-4256-a01d-4d8f8b1c619f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:32.339938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 537/1068] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:32.390103 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf0879-a5a0-42d6-b212-668c865003a9 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:32.427598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852ef39-7335-4c43-9e26-f41c45536243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:32.428243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852ef39-7335-4c43-9e26-f41c45536243 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:32.428519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852ef39-7335-4c43-9e26-f41c45536243 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:32.428789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852ef39-7335-4c43-9e26-f41c45536243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:32.441317 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852ef39-7335-4c43-9e26-f41c45536243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:32.442824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852ef39-7335-4c43-9e26-f41c45536243 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6852ef39-7335-4c43-9e26-f41c45536243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:32.458237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:32.459515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:32.460645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:32.460645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:32.460645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:32.460645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 532/1069] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:32.465462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefa631-9bef-4256-a01d-4d8f8b1c619f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:32.575256 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfefa631-9bef-4256-a01d-4d8f8b1c619f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:32.613601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfc21c-4185-4db7-a64e-d3fb81ff8faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:32.614241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfc21c-4185-4db7-a64e-d3fb81ff8faa None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:32.614517 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfc21c-4185-4db7-a64e-d3fb81ff8faa None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:32.614787 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfc21c-4185-4db7-a64e-d3fb81ff8faa None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:32.615428 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfc21c-4185-4db7-a64e-d3fb81ff8faa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:32.615729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfc21c-4185-4db7-a64e-d3fb81ff8faa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:32.617075 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfc21c-4185-4db7-a64e-d3fb81ff8faa None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:32.619432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:32.630502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 538/1070] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:32.635777 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852ef39-7335-4c43-9e26-f41c45536243 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:32.665863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:32.740736 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852ef39-7335-4c43-9e26-f41c45536243 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:32.772891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2cbc8-5d12-49e9-900f-dd16e8de2d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:32.773510 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2cbc8-5d12-49e9-900f-dd16e8de2d14 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:32.773786 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2cbc8-5d12-49e9-900f-dd16e8de2d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:32.774054 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba2cbc8-5d12-49e9-900f-dd16e8de2d14 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:32.781876 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba2cbc8-5d12-49e9-900f-dd16e8de2d14 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:32.783342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba2cbc8-5d12-49e9-900f-dd16e8de2d14 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6ba2cbc8-5d12-49e9-900f-dd16e8de2d14 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:12:32.794228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:32.795948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:32.796966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:32.796966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:32.796966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:12:32.796966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:12:32.796966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:12:32.796966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:12:32.796966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:12:32.796966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:12:32.796966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:12:32.796966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:32.796966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 533/1071] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Sep 27 17:12:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:32.801859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfc21c-4185-4db7-a64e-d3fb81ff8faa None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:32.897558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfc21c-4185-4db7-a64e-d3fb81ff8faa None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:32.930319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e8024-f136-4020-b959-5e9c0ed7798b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:32.930909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e8024-f136-4020-b959-5e9c0ed7798b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:32.931178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e8024-f136-4020-b959-5e9c0ed7798b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:32.931442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e8024-f136-4020-b959-5e9c0ed7798b None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:32.932152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e8024-f136-4020-b959-5e9c0ed7798b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:32.932446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e8024-f136-4020-b959-5e9c0ed7798b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:32.933844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e8024-f136-4020-b959-5e9c0ed7798b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:32.951002 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 539/1072] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Sep 27 17:12:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:32.956818 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2cbc8-5d12-49e9-900f-dd16e8de2d14 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:33.037352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:33.055116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba2cbc8-5d12-49e9-900f-dd16e8de2d14 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:33.086725 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2912c-bd05-481f-bd38-44c15151c744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:33.087332 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2912c-bd05-481f-bd38-44c15151c744 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:33.087604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2912c-bd05-481f-bd38-44c15151c744 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:33.087868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2912c-bd05-481f-bd38-44c15151c744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:33.098501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2912c-bd05-481f-bd38-44c15151c744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:33.099922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2912c-bd05-481f-bd38-44c15151c744 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-02d2912c-bd05-481f-bd38-44c15151c744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:33.114821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:33.115806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:33.116725 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:33.116725 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:33.116725 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:33.116725 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 534/1073] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:33.121717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e8024-f136-4020-b959-5e9c0ed7798b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:33.175787 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:33.218962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e8024-f136-4020-b959-5e9c0ed7798b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:33.250833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a653e-e238-4ab4-8177-02e51df399e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:33.251416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a653e-e238-4ab4-8177-02e51df399e8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:33.251685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a653e-e238-4ab4-8177-02e51df399e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:33.251954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a653e-e238-4ab4-8177-02e51df399e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:33.252580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a653e-e238-4ab4-8177-02e51df399e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:33.252874 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a653e-e238-4ab4-8177-02e51df399e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:33.254244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a653e-e238-4ab4-8177-02e51df399e8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:33.265410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 540/1074] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:33.271219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2912c-bd05-481f-bd38-44c15151c744 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:33.375910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2912c-bd05-481f-bd38-44c15151c744 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:33.408937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbeaf12-e99a-4d07-ad65-206f67500485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:33.409554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbeaf12-e99a-4d07-ad65-206f67500485 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:33.409831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbeaf12-e99a-4d07-ad65-206f67500485 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:33.410119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbeaf12-e99a-4d07-ad65-206f67500485 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:33.411941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:33.417343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbeaf12-e99a-4d07-ad65-206f67500485 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:33.419025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbeaf12-e99a-4d07-ad65-206f67500485 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6fbeaf12-e99a-4d07-ad65-206f67500485 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:12:33.430527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:33.431655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:33.432567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:12:33.432567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:12:33.432567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:12:33.432567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:12:33.432567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:12:33.432567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:12:33.432567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:33.432567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:33.432567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 535/1075] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:12:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:33.437681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a653e-e238-4ab4-8177-02e51df399e8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:33.533743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a653e-e238-4ab4-8177-02e51df399e8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:33.546645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:33.566789 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d162100-47f8-4f7f-865e-931c996282d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:33.567381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d162100-47f8-4f7f-865e-931c996282d3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:33.567705 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d162100-47f8-4f7f-865e-931c996282d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:33.567926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d162100-47f8-4f7f-865e-931c996282d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:33.568608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d162100-47f8-4f7f-865e-931c996282d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:33.568911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d162100-47f8-4f7f-865e-931c996282d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:33.570355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d162100-47f8-4f7f-865e-931c996282d3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:33.582504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 541/1076] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:12:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:33.588416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbeaf12-e99a-4d07-ad65-206f67500485 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:33.685186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbeaf12-e99a-4d07-ad65-206f67500485 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:33.718638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10ed90-d74c-4ad3-ad61-d19d06740211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:33.719289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10ed90-d74c-4ad3-ad61-d19d06740211 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:33.719565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10ed90-d74c-4ad3-ad61-d19d06740211 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:33.719833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10ed90-d74c-4ad3-ad61-d19d06740211 None admin] PATH_INFO: `/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/5537185f42474cfd86fca70fa8beddd2/roles/a2cb336de6ca492ab46d7dbad02e31a0` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:33.721952 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10ed90-d74c-4ad3-ad61-d19d06740211 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1f2574a4344711a56722bca143ae6c, user_id=5537185f42474cfd86fca70fa8beddd2, role_id=a2cb336de6ca492ab46d7dbad02e31a0)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:33.723406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10ed90-d74c-4ad3-ad61-d19d06740211 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:33.736107 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 536/1077] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:12:33 2022] PUT /identity/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/5537185f42474cfd86fca70fa8beddd2/roles/a2cb336de6ca492ab46d7dbad02e31a0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:12:34.243150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01079a-f4aa-4bfa-8afa-7ee8bd611b8a None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:34.243432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01079a-f4aa-4bfa-8afa-7ee8bd611b8a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:34.243707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01079a-f4aa-4bfa-8afa-7ee8bd611b8a None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:34.245057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 542/1078] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:34.250383 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c090adb-0790-4660-92d4-8a82d9a63a71 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:34.250673 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c090adb-0790-4660-92d4-8a82d9a63a71 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:34.250948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c090adb-0790-4660-92d4-8a82d9a63a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:34.269133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5c090adb-0790-4660-92d4-8a82d9a63a71 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:34.270339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5c090adb-0790-4660-92d4-8a82d9a63a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:34.320727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:34.339524 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 537/1079] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:34.605146 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01079a-f4aa-4bfa-8afa-7ee8bd611b8a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:34.702862 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01079a-f4aa-4bfa-8afa-7ee8bd611b8a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:34.735195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3c27e-6c6d-4d5e-91e6-3f89bc605945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:34.735804 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3c27e-6c6d-4d5e-91e6-3f89bc605945 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:34.736078 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3c27e-6c6d-4d5e-91e6-3f89bc605945 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:34.736346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3c27e-6c6d-4d5e-91e6-3f89bc605945 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:34.746017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3c27e-6c6d-4d5e-91e6-3f89bc605945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7dc3c27e-6c6d-4d5e-91e6-3f89bc605945 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:12:34.751311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:34.752438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:34.753324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:34.753324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:34.753324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:34.753324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:34.753324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:12:34.753324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:12:34.753324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:12:34.753324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:12:34.753324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:12:34.753324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:12:34.753324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:12:34.753324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:34.753324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 543/1080] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:12:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:34.758427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c090adb-0790-4660-92d4-8a82d9a63a71 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:34.853251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c090adb-0790-4660-92d4-8a82d9a63a71 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:34.886998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a1170-13f8-45cf-9bae-5344c9801a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:34.887529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a1170-13f8-45cf-9bae-5344c9801a32 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:34.887803 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a1170-13f8-45cf-9bae-5344c9801a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:34.888067 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a1170-13f8-45cf-9bae-5344c9801a32 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:34.889261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a1170-13f8-45cf-9bae-5344c9801a32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:34.889557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a1170-13f8-45cf-9bae-5344c9801a32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:34.890942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427a1170-13f8-45cf-9bae-5344c9801a32 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:34.899189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 538/1081] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:12:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:34.905219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3c27e-6c6d-4d5e-91e6-3f89bc605945 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:34.940000 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:35.005220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3c27e-6c6d-4d5e-91e6-3f89bc605945 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:35.038577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec8785-c97e-4ef3-9bd4-a6eee203ac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:35.039197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec8785-c97e-4ef3-9bd4-a6eee203ac9b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:35.039473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec8785-c97e-4ef3-9bd4-a6eee203ac9b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:35.039744 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec8785-c97e-4ef3-9bd4-a6eee203ac9b None admin] PATH_INFO: `/v3/users/e55cf7c6da094849945a1b43ae15d533` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:35.040860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec8785-c97e-4ef3-9bd4-a6eee203ac9b None admin] RBAC: Authorizing `identity:get_user(user_id=e55cf7c6da094849945a1b43ae15d533)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:35.042302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec8785-c97e-4ef3-9bd4-a6eee203ac9b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:35.043876 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 544/1082] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:12:34 2022] GET /identity/v3/users/e55cf7c6da094849945a1b43ae15d533 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:35.050458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a1170-13f8-45cf-9bae-5344c9801a32 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:35.095064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:35.146865 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a1170-13f8-45cf-9bae-5344c9801a32 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:35.179072 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2514ba-5090-4c2d-9060-8e99b8bfb35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:35.179686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2514ba-5090-4c2d-9060-8e99b8bfb35e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:35.179961 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2514ba-5090-4c2d-9060-8e99b8bfb35e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:35.180232 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2514ba-5090-4c2d-9060-8e99b8bfb35e None admin] PATH_INFO: `/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:35.181326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2514ba-5090-4c2d-9060-8e99b8bfb35e None admin] RBAC: Authorizing `identity:get_project(project_id=2ec9f3d4fe3547faa80e681ff40e967c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:35.182789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2514ba-5090-4c2d-9060-8e99b8bfb35e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:35.184407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 539/1083] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:12:35 2022] GET /identity/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:35.190776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec8785-c97e-4ef3-9bd4-a6eee203ac9b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:35.287452 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec8785-c97e-4ef3-9bd4-a6eee203ac9b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:35.288954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:35.328545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b617779-bccf-4f2e-9e18-7c2dafc00bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:35.329200 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b617779-bccf-4f2e-9e18-7c2dafc00bb2 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:35.329475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b617779-bccf-4f2e-9e18-7c2dafc00bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:35.329744 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b617779-bccf-4f2e-9e18-7c2dafc00bb2 None admin] PATH_INFO: `/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c/users/e55cf7c6da094849945a1b43ae15d533/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:35.331652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b617779-bccf-4f2e-9e18-7c2dafc00bb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ec9f3d4fe3547faa80e681ff40e967c, user_id=e55cf7c6da094849945a1b43ae15d533, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:35.333081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b617779-bccf-4f2e-9e18-7c2dafc00bb2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:35.346029 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 545/1084] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:12:35 2022] PUT /identity/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c/users/e55cf7c6da094849945a1b43ae15d533/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:12:35.859482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1018bee-258b-406c-91e9-013deb8037bc None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:35.860058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1018bee-258b-406c-91e9-013deb8037bc None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:35.860058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1018bee-258b-406c-91e9-013deb8037bc None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:35.861368 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 540/1085] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:35.866703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763b50b-7dd3-4cb0-ad40-2ad794ffc680 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:35.867137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763b50b-7dd3-4cb0-ad40-2ad794ffc680 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:35.867248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763b50b-7dd3-4cb0-ad40-2ad794ffc680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:35.885224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1763b50b-7dd3-4cb0-ad40-2ad794ffc680 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:35.886419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1763b50b-7dd3-4cb0-ad40-2ad794ffc680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:35.949968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 546/1086] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:36.218112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1018bee-258b-406c-91e9-013deb8037bc None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:36.316107 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1018bee-258b-406c-91e9-013deb8037bc None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:36.329584 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:36.349741 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d1ebb-d434-4bd0-a5aa-dba4a2fb9207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:36.350386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d1ebb-d434-4bd0-a5aa-dba4a2fb9207 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:36.350663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d1ebb-d434-4bd0-a5aa-dba4a2fb9207 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:36.350930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d1ebb-d434-4bd0-a5aa-dba4a2fb9207 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:36.361373 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d1ebb-d434-4bd0-a5aa-dba4a2fb9207 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-f98d1ebb-d434-4bd0-a5aa-dba4a2fb9207 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:12:36.366098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:36.367269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:36.368186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:36.368186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:36.368186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:36.368186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:36.368186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:12:36.368186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:12:36.368186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:12:36.368186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:12:36.368186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:12:36.368186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:12:36.368186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 27 17:12:36.368186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:36.368186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 541/1087] 10.222.0.55 () {56 vars in 1120 bytes} [Tue Sep 27 17:12:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:36.373693 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763b50b-7dd3-4cb0-ad40-2ad794ffc680 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:36.459719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:36.470696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763b50b-7dd3-4cb0-ad40-2ad794ffc680 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:36.504920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa98ce3-baba-4faf-b308-94068f88e97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:36.505512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa98ce3-baba-4faf-b308-94068f88e97a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:36.505786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa98ce3-baba-4faf-b308-94068f88e97a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:36.506053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa98ce3-baba-4faf-b308-94068f88e97a None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:36.506740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa98ce3-baba-4faf-b308-94068f88e97a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:36.507036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa98ce3-baba-4faf-b308-94068f88e97a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:36.508381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa98ce3-baba-4faf-b308-94068f88e97a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:36.516155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 547/1088] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Sep 27 17:12:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:36.521871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d1ebb-d434-4bd0-a5aa-dba4a2fb9207 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:36.619030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d1ebb-d434-4bd0-a5aa-dba4a2fb9207 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:36.651820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e140bc-ce21-445b-85de-2c093e6b14b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:36.652434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e140bc-ce21-445b-85de-2c093e6b14b4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:36.652705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e140bc-ce21-445b-85de-2c093e6b14b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:36.652968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e140bc-ce21-445b-85de-2c093e6b14b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:36.662543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e140bc-ce21-445b-85de-2c093e6b14b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:36.663962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e140bc-ce21-445b-85de-2c093e6b14b4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-99e140bc-ce21-445b-85de-2c093e6b14b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:36.677817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:36.679042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:36.680376 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:36.680376 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:36.680376 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:36.682378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 542/1089] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:36.684601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa98ce3-baba-4faf-b308-94068f88e97a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:36.781601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa98ce3-baba-4faf-b308-94068f88e97a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:36.813418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42539067-a382-43ae-80d6-29dc47a5028e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:36.814002 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42539067-a382-43ae-80d6-29dc47a5028e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:36.814304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42539067-a382-43ae-80d6-29dc47a5028e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:36.814571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42539067-a382-43ae-80d6-29dc47a5028e None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:36.815201 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42539067-a382-43ae-80d6-29dc47a5028e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:36.815496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42539067-a382-43ae-80d6-29dc47a5028e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:36.816827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42539067-a382-43ae-80d6-29dc47a5028e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:36.828589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 548/1090] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:36.834257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e140bc-ce21-445b-85de-2c093e6b14b4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:36.861051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:36.930664 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e140bc-ce21-445b-85de-2c093e6b14b4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:36.963478 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0fd46-6c8d-407c-a46b-9daf4192cf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:36.963912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0fd46-6c8d-407c-a46b-9daf4192cf20 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:36.964187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0fd46-6c8d-407c-a46b-9daf4192cf20 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:36.964565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0fd46-6c8d-407c-a46b-9daf4192cf20 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:36.972060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0fd46-6c8d-407c-a46b-9daf4192cf20 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:36.973508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0fd46-6c8d-407c-a46b-9daf4192cf20 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-52e0fd46-6c8d-407c-a46b-9daf4192cf20 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:12:36.985035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:36.986007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:36.987006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:36.987006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:36.987006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:12:36.987006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:12:36.987006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:12:36.987006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:12:36.987006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:12:36.987006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:12:36.987006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 27 17:12:36.987006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:36.987006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 543/1091] 10.222.0.55 () {56 vars in 1093 bytes} [Tue Sep 27 17:12:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:36.991904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42539067-a382-43ae-80d6-29dc47a5028e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:37.007625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:37.088516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42539067-a382-43ae-80d6-29dc47a5028e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:37.121164 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be645d-f270-42ca-a802-39f7f4546eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:37.121753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be645d-f270-42ca-a802-39f7f4546eeb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:37.122026 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be645d-f270-42ca-a802-39f7f4546eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:37.122320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be645d-f270-42ca-a802-39f7f4546eeb None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:37.123628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be645d-f270-42ca-a802-39f7f4546eeb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:37.123931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be645d-f270-42ca-a802-39f7f4546eeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:37.125336 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be645d-f270-42ca-a802-39f7f4546eeb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:37.142708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 549/1092] 10.222.0.55 () {56 vars in 1133 bytes} [Tue Sep 27 17:12:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:37.148473 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0fd46-6c8d-407c-a46b-9daf4192cf20 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:37.232806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:37.249032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0fd46-6c8d-407c-a46b-9daf4192cf20 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:37.281864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f355d78-7c45-4605-9f06-f6b307e9f0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:37.282492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f355d78-7c45-4605-9f06-f6b307e9f0fc None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:37.282770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f355d78-7c45-4605-9f06-f6b307e9f0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:37.283040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f355d78-7c45-4605-9f06-f6b307e9f0fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:37.348873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f355d78-7c45-4605-9f06-f6b307e9f0fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:37.350347 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f355d78-7c45-4605-9f06-f6b307e9f0fc None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:37.434649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:37.450024 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 544/1093] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:12:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 27 17:12:37.455697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be645d-f270-42ca-a802-39f7f4546eeb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:37.510757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8631c-72e2-4463-99e9-b625b927016e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:37.511039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8631c-72e2-4463-99e9-b625b927016e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:37.511323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8631c-72e2-4463-99e9-b625b927016e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:37.512665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 545/1094] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:37.517598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60fd19-8ba6-4df2-9bc7-e9e163f84f6f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:37.517947 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60fd19-8ba6-4df2-9bc7-e9e163f84f6f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:37.518260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60fd19-8ba6-4df2-9bc7-e9e163f84f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:37.525409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:37.537508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7a60fd19-8ba6-4df2-9bc7-e9e163f84f6f None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:37.538731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7a60fd19-8ba6-4df2-9bc7-e9e163f84f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:37.556511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be645d-f270-42ca-a802-39f7f4546eeb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:37.588888 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58d6a6-0167-4049-884f-aef336d31f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:37.589506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58d6a6-0167-4049-884f-aef336d31f64 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:37.589782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58d6a6-0167-4049-884f-aef336d31f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:37.590053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d58d6a6-0167-4049-884f-aef336d31f64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:37.600318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58d6a6-0167-4049-884f-aef336d31f64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:37.601682 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d58d6a6-0167-4049-884f-aef336d31f64 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:37.604832 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 546/1095] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1d58d6a6-0167-4049-884f-aef336d31f64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:12:37.616807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:37.617951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:12:37.619069 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:12:37.619069 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:12:37.619069 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:37.619069 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 550/1096] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:37.623435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60fd19-8ba6-4df2-9bc7-e9e163f84f6f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:37.721768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60fd19-8ba6-4df2-9bc7-e9e163f84f6f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:37.738339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:37.755636 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bc916-c1c2-40e5-b56b-dbf5c9588961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:37.756239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bc916-c1c2-40e5-b56b-dbf5c9588961 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:37.756515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bc916-c1c2-40e5-b56b-dbf5c9588961 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:37.756786 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bc916-c1c2-40e5-b56b-dbf5c9588961 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:37.757424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bc916-c1c2-40e5-b56b-dbf5c9588961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:37.757723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bc916-c1c2-40e5-b56b-dbf5c9588961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:37.759117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bc916-c1c2-40e5-b56b-dbf5c9588961 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:37.770532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 547/1097] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:37.776106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58d6a6-0167-4049-884f-aef336d31f64 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:37.871871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d58d6a6-0167-4049-884f-aef336d31f64 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:37.873880 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bc916-c1c2-40e5-b56b-dbf5c9588961 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:37.894713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:37.906219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e458809-8d70-409d-99c2-21a1678ec4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:37.906832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e458809-8d70-409d-99c2-21a1678ec4fa None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:37.907104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e458809-8d70-409d-99c2-21a1678ec4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:37.907365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e458809-8d70-409d-99c2-21a1678ec4fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:37.913763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e458809-8d70-409d-99c2-21a1678ec4fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:37.915219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e458809-8d70-409d-99c2-21a1678ec4fa None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-8e458809-8d70-409d-99c2-21a1678ec4fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:12:37.926109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:37.927257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:37.928191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:12:37.928191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:12:37.928191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:12:37.928191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:12:37.928191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:12:37.928191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:12:37.928191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:12:37.928191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:12:37.928191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 551/1098] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:12:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:12:37.932870 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e458809-8d70-409d-99c2-21a1678ec4fa None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:37.976806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bc916-c1c2-40e5-b56b-dbf5c9588961 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:38.009449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77798287-d33c-4a43-af25-64986e1b16c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:38.010052 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77798287-d33c-4a43-af25-64986e1b16c1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:38.010356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77798287-d33c-4a43-af25-64986e1b16c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:38.010620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77798287-d33c-4a43-af25-64986e1b16c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:38.021251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77798287-d33c-4a43-af25-64986e1b16c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-77798287-d33c-4a43-af25-64986e1b16c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:12:38.025891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:12:38.026899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:12:38.027768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:12:38.027768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:12:38.027768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:12:38.027768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:12:38.027768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:12:38.027768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:12:38.027768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:12:38.027768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:12:38.027768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:12:38.027768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:12:38.027768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:12:38.027768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:12:38.027768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 548/1099] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:12:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:12:38.031672 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e458809-8d70-409d-99c2-21a1678ec4fa None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:38.033786 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77798287-d33c-4a43-af25-64986e1b16c1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:38.065831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea3a6a-3b5b-4337-8161-d8f14f7cb620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:38.066475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea3a6a-3b5b-4337-8161-d8f14f7cb620 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:38.066763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea3a6a-3b5b-4337-8161-d8f14f7cb620 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:38.067030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea3a6a-3b5b-4337-8161-d8f14f7cb620 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:38.067734 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea3a6a-3b5b-4337-8161-d8f14f7cb620 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:38.068037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea3a6a-3b5b-4337-8161-d8f14f7cb620 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:38.069485 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea3a6a-3b5b-4337-8161-d8f14f7cb620 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:38.079555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:38.085124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 552/1100] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:12:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:38.091830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea3a6a-3b5b-4337-8161-d8f14f7cb620 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:38.137397 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:38.160387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77798287-d33c-4a43-af25-64986e1b16c1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:38.191920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea3a6a-3b5b-4337-8161-d8f14f7cb620 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:38.195272 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1763010-eb14-466f-8648-ab416471dbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:38.195871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1763010-eb14-466f-8648-ab416471dbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:38.196143 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1763010-eb14-466f-8648-ab416471dbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:38.196408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1763010-eb14-466f-8648-ab416471dbd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:38.197060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1763010-eb14-466f-8648-ab416471dbd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:38.197356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1763010-eb14-466f-8648-ab416471dbd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:38.198740 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1763010-eb14-466f-8648-ab416471dbd0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:38.206522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 549/1101] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:12:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:38.212478 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1763010-eb14-466f-8648-ab416471dbd0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:38.225488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ec716-6388-4b76-b683-5000666857a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:38.226114 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ec716-6388-4b76-b683-5000666857a0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:38.226397 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ec716-6388-4b76-b683-5000666857a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:38.226670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ec716-6388-4b76-b683-5000666857a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:38.227464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ec716-6388-4b76-b683-5000666857a0 None admin] RBAC: Adding query filter params (role.id=a2cb336de6ca492ab46d7dbad02e31a0, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=5537185f42474cfd86fca70fa8beddd2) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:38.227769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ec716-6388-4b76-b683-5000666857a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:38.229245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ec716-6388-4b76-b683-5000666857a0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:38.235261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 553/1102] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:12:38 2022] GET /identity/v3/role_assignments?user.id=5537185f42474cfd86fca70fa8beddd2&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=a2cb336de6ca492ab46d7dbad02e31a0 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:38.324882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:38.329386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1763010-eb14-466f-8648-ab416471dbd0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:38.366685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e9345-6e7e-4700-99a6-2bf1932aedf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:38.367319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e9345-6e7e-4700-99a6-2bf1932aedf3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:38.367592 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e9345-6e7e-4700-99a6-2bf1932aedf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:38.367859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e9345-6e7e-4700-99a6-2bf1932aedf3 None admin] PATH_INFO: `/v3/users/e55cf7c6da094849945a1b43ae15d533` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:38.369004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e9345-6e7e-4700-99a6-2bf1932aedf3 None admin] RBAC: Authorizing `identity:get_user(user_id=e55cf7c6da094849945a1b43ae15d533)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:38.370495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e9345-6e7e-4700-99a6-2bf1932aedf3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:38.372184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 550/1103] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:12:38 2022] GET /identity/v3/users/e55cf7c6da094849945a1b43ae15d533 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:38.378106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ec716-6388-4b76-b683-5000666857a0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:38.480865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ec716-6388-4b76-b683-5000666857a0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:38.489511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:38.514561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e5c6d-dd0f-4cb4-9770-75b54eeea558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:38.515183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e5c6d-dd0f-4cb4-9770-75b54eeea558 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:38.515457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e5c6d-dd0f-4cb4-9770-75b54eeea558 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:38.515724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e5c6d-dd0f-4cb4-9770-75b54eeea558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:38.585108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e5c6d-dd0f-4cb4-9770-75b54eeea558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:38.586581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e5c6d-dd0f-4cb4-9770-75b54eeea558 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:38.683364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 554/1104] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:12:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:12:38.689503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e9345-6e7e-4700-99a6-2bf1932aedf3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:38.791710 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e9345-6e7e-4700-99a6-2bf1932aedf3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:38.825080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ace417-b5ba-4a8c-be79-dc74c7e407d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:38.825706 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ace417-b5ba-4a8c-be79-dc74c7e407d7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:38.826061 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ace417-b5ba-4a8c-be79-dc74c7e407d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:38.826342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ace417-b5ba-4a8c-be79-dc74c7e407d7 None admin] PATH_INFO: `/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:38.827440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ace417-b5ba-4a8c-be79-dc74c7e407d7 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec9f3d4fe3547faa80e681ff40e967c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:38.828851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ace417-b5ba-4a8c-be79-dc74c7e407d7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:38.830766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 551/1105] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:12:38 2022] GET /identity/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:38.836945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e5c6d-dd0f-4cb4-9770-75b54eeea558 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:38.847818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:38.937388 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e5c6d-dd0f-4cb4-9770-75b54eeea558 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:38.973148 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c9349-4938-4e47-a5d4-302dd603a0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:38.973753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c9349-4938-4e47-a5d4-302dd603a0ab None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:38.974030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c9349-4938-4e47-a5d4-302dd603a0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:38.974326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c9349-4938-4e47-a5d4-302dd603a0ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:38.975051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c9349-4938-4e47-a5d4-302dd603a0ab None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=2ec9f3d4fe3547faa80e681ff40e967c, user.id=e55cf7c6da094849945a1b43ae15d533) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:38.975362 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c9349-4938-4e47-a5d4-302dd603a0ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:38.976766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c9349-4938-4e47-a5d4-302dd603a0ab None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:38.983291 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 555/1106] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:12:38 2022] GET /identity/v3/role_assignments?user.id=e55cf7c6da094849945a1b43ae15d533&scope.project.id=2ec9f3d4fe3547faa80e681ff40e967c&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:41.191038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf4ab4-e2fd-482e-a815-f26e2cd9a3a7 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:41.191593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf4ab4-e2fd-482e-a815-f26e2cd9a3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:41.191593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf4ab4-e2fd-482e-a815-f26e2cd9a3a7 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:41.192914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 552/1107] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:41.198180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c7b77-fda0-4e0f-ad49-b4d88083fdfd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:41.198546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c7b77-fda0-4e0f-ad49-b4d88083fdfd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:41.198730 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c7b77-fda0-4e0f-ad49-b4d88083fdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:41.217029 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-608c7b77-fda0-4e0f-ad49-b4d88083fdfd None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:41.218248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-608c7b77-fda0-4e0f-ad49-b4d88083fdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:41.282085 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 556/1108] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:41.554511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf4ab4-e2fd-482e-a815-f26e2cd9a3a7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:41.588710 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:41.653280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf4ab4-e2fd-482e-a815-f26e2cd9a3a7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:41.687708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d3c2d-325e-4179-9c77-0979ae37c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:41.688314 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d3c2d-325e-4179-9c77-0979ae37c816 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:41.688588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d3c2d-325e-4179-9c77-0979ae37c816 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:41.688854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d3c2d-325e-4179-9c77-0979ae37c816 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:41.689936 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d3c2d-325e-4179-9c77-0979ae37c816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:41.691391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d3c2d-325e-4179-9c77-0979ae37c816 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:41.692941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 553/1109] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:41.698687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c7b77-fda0-4e0f-ad49-b4d88083fdfd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:41.700817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:41.795403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c7b77-fda0-4e0f-ad49-b4d88083fdfd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:41.827241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df397d-e2a4-471d-ac1c-d3da45ef8295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:41.827770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df397d-e2a4-471d-ac1c-d3da45ef8295 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:41.828049 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df397d-e2a4-471d-ac1c-d3da45ef8295 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:41.828321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df397d-e2a4-471d-ac1c-d3da45ef8295 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:41.828959 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df397d-e2a4-471d-ac1c-d3da45ef8295 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:41.830392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df397d-e2a4-471d-ac1c-d3da45ef8295 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:41.850790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 557/1110] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:12:41 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:44.018301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d4469-74bc-4e30-ab06-2b71a31daf42 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:44.018901 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d4469-74bc-4e30-ab06-2b71a31daf42 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:44.018901 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d4469-74bc-4e30-ab06-2b71a31daf42 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:44.020158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 554/1111] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:44.025447 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51eb2-89a2-4481-99c8-d7efc8d2ff29 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:44.025880 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51eb2-89a2-4481-99c8-d7efc8d2ff29 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:44.025983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51eb2-89a2-4481-99c8-d7efc8d2ff29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:44.044081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-92b51eb2-89a2-4481-99c8-d7efc8d2ff29 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:44.045265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-92b51eb2-89a2-4481-99c8-d7efc8d2ff29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:44.096187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:44.110244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 558/1112] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:44.395346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d4469-74bc-4e30-ab06-2b71a31daf42 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:44.492442 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d4469-74bc-4e30-ab06-2b71a31daf42 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:44.493886 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:44.525728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2172d9b-fbac-4f6a-84ad-97061ca774ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:44.526362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2172d9b-fbac-4f6a-84ad-97061ca774ed None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:44.526633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2172d9b-fbac-4f6a-84ad-97061ca774ed None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:44.526897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2172d9b-fbac-4f6a-84ad-97061ca774ed None admin] PATH_INFO: `/v3/roles/dfb71501dfed4ce48cf8cb3559cee9d6` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:44.528241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2172d9b-fbac-4f6a-84ad-97061ca774ed None admin] RBAC: Authorizing `identity:get_role(role_id=dfb71501dfed4ce48cf8cb3559cee9d6)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:44.530267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2172d9b-fbac-4f6a-84ad-97061ca774ed None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:44.531551 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 555/1113] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:12:44 2022] GET /identity/v3/roles/dfb71501dfed4ce48cf8cb3559cee9d6 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:44.537327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b51eb2-89a2-4481-99c8-d7efc8d2ff29 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:44.631170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b51eb2-89a2-4481-99c8-d7efc8d2ff29 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:44.663286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74191e-8807-4bb1-b39a-02aa728b9b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:44.663889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74191e-8807-4bb1-b39a-02aa728b9b68 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:44.664160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74191e-8807-4bb1-b39a-02aa728b9b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:44.664422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74191e-8807-4bb1-b39a-02aa728b9b68 None admin] PATH_INFO: `/v3/users/44d8f5a694a64f8fbb7e67731d20d624` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:44.677158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74191e-8807-4bb1-b39a-02aa728b9b68 None admin] RBAC: Authorizing `identity:get_user(user_id=44d8f5a694a64f8fbb7e67731d20d624)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:44.678598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74191e-8807-4bb1-b39a-02aa728b9b68 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:44.680160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 559/1114] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:12:44 2022] GET /identity/v3/users/44d8f5a694a64f8fbb7e67731d20d624 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:44.686203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2172d9b-fbac-4f6a-84ad-97061ca774ed None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:44.785753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2172d9b-fbac-4f6a-84ad-97061ca774ed None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:44.818955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bfcad-7a44-4512-b0f7-754cfa6205ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:44.819549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bfcad-7a44-4512-b0f7-754cfa6205ca None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:44.819826 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bfcad-7a44-4512-b0f7-754cfa6205ca None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:44.820095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bfcad-7a44-4512-b0f7-754cfa6205ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:44.857840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:44.886356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bfcad-7a44-4512-b0f7-754cfa6205ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:44.887731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684bfcad-7a44-4512-b0f7-754cfa6205ca None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:44.981173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 556/1115] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:12:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:12:44.987410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74191e-8807-4bb1-b39a-02aa728b9b68 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:45.035372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:45.088324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74191e-8807-4bb1-b39a-02aa728b9b68 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:45.131313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb5cb9-33ae-4dbd-8ea7-b0462df9be95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:45.131934 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb5cb9-33ae-4dbd-8ea7-b0462df9be95 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:45.132206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb5cb9-33ae-4dbd-8ea7-b0462df9be95 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:45.132471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb5cb9-33ae-4dbd-8ea7-b0462df9be95 None admin] PATH_INFO: `/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:45.133644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb5cb9-33ae-4dbd-8ea7-b0462df9be95 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec9f3d4fe3547faa80e681ff40e967c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:45.135087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb5cb9-33ae-4dbd-8ea7-b0462df9be95 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:45.136826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 560/1116] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:12:44 2022] GET /identity/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:12:45.144816 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bfcad-7a44-4512-b0f7-754cfa6205ca None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:45.211558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:45.242299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bfcad-7a44-4512-b0f7-754cfa6205ca None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:45.274792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a2136-5828-4f02-bc46-3d35d9a8259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:45.275378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a2136-5828-4f02-bc46-3d35d9a8259e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:45.275650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a2136-5828-4f02-bc46-3d35d9a8259e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:45.275916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a2136-5828-4f02-bc46-3d35d9a8259e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:45.276632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a2136-5828-4f02-bc46-3d35d9a8259e None admin] RBAC: Adding query filter params (role.id=dfb71501dfed4ce48cf8cb3559cee9d6, scope.project.id=2ec9f3d4fe3547faa80e681ff40e967c, user.id=44d8f5a694a64f8fbb7e67731d20d624) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:45.276937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a2136-5828-4f02-bc46-3d35d9a8259e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:45.278375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a2136-5828-4f02-bc46-3d35d9a8259e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:45.283566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 557/1117] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:12:45 2022] GET /identity/v3/role_assignments?user.id=44d8f5a694a64f8fbb7e67731d20d624&scope.project.id=2ec9f3d4fe3547faa80e681ff40e967c&role.id=dfb71501dfed4ce48cf8cb3559cee9d6 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:47.431435 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a153d-f0d3-4169-9926-b1f8eda1183c None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:47.431956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a153d-f0d3-4169-9926-b1f8eda1183c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:47.431956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a153d-f0d3-4169-9926-b1f8eda1183c None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:47.433295 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 561/1118] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:47.438342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b22ee-ca49-4762-89f5-ec28f4163ce7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:47.438763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b22ee-ca49-4762-89f5-ec28f4163ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:47.438885 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b22ee-ca49-4762-89f5-ec28f4163ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:47.456973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ac3b22ee-ca49-4762-89f5-ec28f4163ce7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:47.458161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ac3b22ee-ca49-4762-89f5-ec28f4163ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:47.530139 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 558/1119] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:47.797038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a153d-f0d3-4169-9926-b1f8eda1183c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:47.895602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a153d-f0d3-4169-9926-b1f8eda1183c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:47.908843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:47.928540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18297c9-9a9a-4b21-ac8a-b62d2023e1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:47.929151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18297c9-9a9a-4b21-ac8a-b62d2023e1df None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:47.929426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18297c9-9a9a-4b21-ac8a-b62d2023e1df None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:47.929693 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18297c9-9a9a-4b21-ac8a-b62d2023e1df None admin] PATH_INFO: `/v3/roles/dfb71501dfed4ce48cf8cb3559cee9d6` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:47.931081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18297c9-9a9a-4b21-ac8a-b62d2023e1df None admin] RBAC: Authorizing `identity:get_role(role_id=dfb71501dfed4ce48cf8cb3559cee9d6)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:47.932430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18297c9-9a9a-4b21-ac8a-b62d2023e1df None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:47.933735 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 562/1120] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:12:47 2022] GET /identity/v3/roles/dfb71501dfed4ce48cf8cb3559cee9d6 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:47.940093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b22ee-ca49-4762-89f5-ec28f4163ce7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:47.999951 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:48.036354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b22ee-ca49-4762-89f5-ec28f4163ce7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:48.068795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965bbcd-3922-465a-ada4-713d045f696b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:48.069351 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965bbcd-3922-465a-ada4-713d045f696b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:48.069626 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965bbcd-3922-465a-ada4-713d045f696b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:48.069895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965bbcd-3922-465a-ada4-713d045f696b None admin] PATH_INFO: `/v3/users/44d8f5a694a64f8fbb7e67731d20d624` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:48.071025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965bbcd-3922-465a-ada4-713d045f696b None admin] RBAC: Authorizing `identity:get_user(user_id=44d8f5a694a64f8fbb7e67731d20d624)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:48.072440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965bbcd-3922-465a-ada4-713d045f696b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:48.073996 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 559/1121] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:12:47 2022] GET /identity/v3/users/44d8f5a694a64f8fbb7e67731d20d624 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:48.080263 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18297c9-9a9a-4b21-ac8a-b62d2023e1df None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:48.176181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18297c9-9a9a-4b21-ac8a-b62d2023e1df None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:48.208208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2cc133-91d6-4404-bdf4-a30cb2e2af97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:48.208807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cc133-91d6-4404-bdf4-a30cb2e2af97 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:48.209081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cc133-91d6-4404-bdf4-a30cb2e2af97 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:48.209351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2cc133-91d6-4404-bdf4-a30cb2e2af97 None admin] PATH_INFO: `/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:48.210466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2cc133-91d6-4404-bdf4-a30cb2e2af97 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec9f3d4fe3547faa80e681ff40e967c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:48.211870 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2cc133-91d6-4404-bdf4-a30cb2e2af97 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:48.213462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 563/1122] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:12:48 2022] GET /identity/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:48.220007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965bbcd-3922-465a-ada4-713d045f696b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:48.315866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965bbcd-3922-465a-ada4-713d045f696b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:48.344751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:48.349984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6e04e-6bd6-494e-b50c-e19933612175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:48.350659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6e04e-6bd6-494e-b50c-e19933612175 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:48.350939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6e04e-6bd6-494e-b50c-e19933612175 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:48.351212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6e04e-6bd6-494e-b50c-e19933612175 None admin] PATH_INFO: `/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c/users/44d8f5a694a64f8fbb7e67731d20d624/roles/dfb71501dfed4ce48cf8cb3559cee9d6` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:48.353317 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6e04e-6bd6-494e-b50c-e19933612175 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ec9f3d4fe3547faa80e681ff40e967c, user_id=44d8f5a694a64f8fbb7e67731d20d624, role_id=dfb71501dfed4ce48cf8cb3559cee9d6)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:48.355063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6e04e-6bd6-494e-b50c-e19933612175 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:48.367036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 560/1123] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:12:48 2022] PUT /identity/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c/users/44d8f5a694a64f8fbb7e67731d20d624/roles/dfb71501dfed4ce48cf8cb3559cee9d6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:12:50.474518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b247306-6d10-4738-81c8-c977138f1a83 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:50.475042 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b247306-6d10-4738-81c8-c977138f1a83 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:50.475136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b247306-6d10-4738-81c8-c977138f1a83 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:50.476394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 564/1124] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:50.482471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4cba2-5985-4375-9686-d81c33b120f2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:50.482870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4cba2-5985-4375-9686-d81c33b120f2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:50.483021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4cba2-5985-4375-9686-d81c33b120f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:50.501283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-13a4cba2-5985-4375-9686-d81c33b120f2 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:50.502503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-13a4cba2-5985-4375-9686-d81c33b120f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:50.565868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 561/1125] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:50.828892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b247306-6d10-4738-81c8-c977138f1a83 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:50.877562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:50.929295 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b247306-6d10-4738-81c8-c977138f1a83 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:50.962221 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c937a-b2f1-42e4-b26e-9ca1c5759252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:50.962826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c937a-b2f1-42e4-b26e-9ca1c5759252 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:50.963098 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c937a-b2f1-42e4-b26e-9ca1c5759252 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:50.963374 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c937a-b2f1-42e4-b26e-9ca1c5759252 None admin] PATH_INFO: `/v3/roles/dfb71501dfed4ce48cf8cb3559cee9d6` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:50.964720 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c937a-b2f1-42e4-b26e-9ca1c5759252 None admin] RBAC: Authorizing `identity:get_role(role_id=dfb71501dfed4ce48cf8cb3559cee9d6)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:50.966120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c937a-b2f1-42e4-b26e-9ca1c5759252 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:50.967425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 565/1126] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:12:50 2022] GET /identity/v3/roles/dfb71501dfed4ce48cf8cb3559cee9d6 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:50.973246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4cba2-5985-4375-9686-d81c33b120f2 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:51.074303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4cba2-5985-4375-9686-d81c33b120f2 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:51.097047 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:51.110308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2a7c9-fd46-4270-9b64-6fb8c010c3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:51.110927 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2a7c9-fd46-4270-9b64-6fb8c010c3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:51.111206 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2a7c9-fd46-4270-9b64-6fb8c010c3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:51.111478 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2a7c9-fd46-4270-9b64-6fb8c010c3e0 None admin] PATH_INFO: `/v3/users/44d8f5a694a64f8fbb7e67731d20d624` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:51.112586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2a7c9-fd46-4270-9b64-6fb8c010c3e0 None admin] RBAC: Authorizing `identity:get_user(user_id=44d8f5a694a64f8fbb7e67731d20d624)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:51.114012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2a7c9-fd46-4270-9b64-6fb8c010c3e0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:51.115645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 562/1127] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:12:50 2022] GET /identity/v3/users/44d8f5a694a64f8fbb7e67731d20d624 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:51.121563 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c937a-b2f1-42e4-b26e-9ca1c5759252 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:51.219169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c937a-b2f1-42e4-b26e-9ca1c5759252 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:51.235793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:51.252079 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5673c-494b-44e6-972b-e6797f055f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:51.252666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5673c-494b-44e6-972b-e6797f055f74 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:51.252946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5673c-494b-44e6-972b-e6797f055f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:51.253215 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5673c-494b-44e6-972b-e6797f055f74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:51.318833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5673c-494b-44e6-972b-e6797f055f74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:51.320257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5673c-494b-44e6-972b-e6797f055f74 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:51.413903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 566/1128] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:12:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:12:51.419811 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2a7c9-fd46-4270-9b64-6fb8c010c3e0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:51.516461 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2a7c9-fd46-4270-9b64-6fb8c010c3e0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:51.547892 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cd4d2-64bb-4042-83fa-e273ddb16f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:51.548498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cd4d2-64bb-4042-83fa-e273ddb16f9a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:51.548771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cd4d2-64bb-4042-83fa-e273ddb16f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:51.549038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cd4d2-64bb-4042-83fa-e273ddb16f9a None admin] PATH_INFO: `/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:51.550181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18cd4d2-64bb-4042-83fa-e273ddb16f9a None admin] RBAC: Authorizing `identity:get_project(project_id=2ec9f3d4fe3547faa80e681ff40e967c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:51.551600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18cd4d2-64bb-4042-83fa-e273ddb16f9a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:51.553201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 563/1129] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:12:51 2022] GET /identity/v3/projects/2ec9f3d4fe3547faa80e681ff40e967c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:51.559185 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5673c-494b-44e6-972b-e6797f055f74 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:51.577651 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:51.655201 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5673c-494b-44e6-972b-e6797f055f74 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:51.687245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034031a-a785-4aa2-b736-3e7f1413e129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:51.687766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034031a-a785-4aa2-b736-3e7f1413e129 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:51.688040 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034031a-a785-4aa2-b736-3e7f1413e129 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:51.688308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034031a-a785-4aa2-b736-3e7f1413e129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:51.689019 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034031a-a785-4aa2-b736-3e7f1413e129 None admin] RBAC: Adding query filter params (role.id=dfb71501dfed4ce48cf8cb3559cee9d6, scope.project.id=2ec9f3d4fe3547faa80e681ff40e967c, user.id=44d8f5a694a64f8fbb7e67731d20d624) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:12:51.689323 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034031a-a785-4aa2-b736-3e7f1413e129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:51.690755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034031a-a785-4aa2-b736-3e7f1413e129 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:51.696227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 567/1130] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:12:51 2022] GET /identity/v3/role_assignments?user.id=44d8f5a694a64f8fbb7e67731d20d624&scope.project.id=2ec9f3d4fe3547faa80e681ff40e967c&role.id=dfb71501dfed4ce48cf8cb3559cee9d6 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:53.845320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e533d-9346-4d37-ab9f-369c19042c5b None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:53.845850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e533d-9346-4d37-ab9f-369c19042c5b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:53.845850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e533d-9346-4d37-ab9f-369c19042c5b None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:53.847233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 564/1131] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:53.852335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd880f0-c8a8-4e4a-8d2a-bd36e0a60b98 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:53.852755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd880f0-c8a8-4e4a-8d2a-bd36e0a60b98 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:53.852879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd880f0-c8a8-4e4a-8d2a-bd36e0a60b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:53.870948 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fcd880f0-c8a8-4e4a-8d2a-bd36e0a60b98 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:53.872630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fcd880f0-c8a8-4e4a-8d2a-bd36e0a60b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:53.937522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 568/1132] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:54.206723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e533d-9346-4d37-ab9f-369c19042c5b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:54.265039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:54.306240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e533d-9346-4d37-ab9f-369c19042c5b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:54.338895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a337f-d7f2-4641-8bc8-c6ef8ef4bed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:54.339501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a337f-d7f2-4641-8bc8-c6ef8ef4bed3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:54.339772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a337f-d7f2-4641-8bc8-c6ef8ef4bed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:54.340033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a337f-d7f2-4641-8bc8-c6ef8ef4bed3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:54.341731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a337f-d7f2-4641-8bc8-c6ef8ef4bed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:54.343169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a337f-d7f2-4641-8bc8-c6ef8ef4bed3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:54.344717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 565/1133] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:54.350513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd880f0-c8a8-4e4a-8d2a-bd36e0a60b98 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:54.364310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:54.445588 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd880f0-c8a8-4e4a-8d2a-bd36e0a60b98 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:54.478664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a510e2-a5cd-4c1c-b673-ae6ed5ac33b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:54.479194 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a510e2-a5cd-4c1c-b673-ae6ed5ac33b3 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:54.479735 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a510e2-a5cd-4c1c-b673-ae6ed5ac33b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:54.480008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a510e2-a5cd-4c1c-b673-ae6ed5ac33b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:54.480647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a510e2-a5cd-4c1c-b673-ae6ed5ac33b3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:54.482111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a510e2-a5cd-4c1c-b673-ae6ed5ac33b3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:54.507291 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 569/1134] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:12:54 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:56.719696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8346ff3-8511-4d88-bde6-e816a3cf3ca5 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:56.720246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8346ff3-8511-4d88-bde6-e816a3cf3ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:56.720246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8346ff3-8511-4d88-bde6-e816a3cf3ca5 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:56.721567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 566/1135] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:12:56.726698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de094bf-d2b0-40fa-8d07-13fa4f7e2431 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:56.727081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de094bf-d2b0-40fa-8d07-13fa4f7e2431 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:56.727238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de094bf-d2b0-40fa-8d07-13fa4f7e2431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:56.745640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7de094bf-d2b0-40fa-8d07-13fa4f7e2431 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:56.747473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7de094bf-d2b0-40fa-8d07-13fa4f7e2431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:56.803847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:56.812963 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 570/1136] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:57.087056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8346ff3-8511-4d88-bde6-e816a3cf3ca5 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:57.185347 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8346ff3-8511-4d88-bde6-e816a3cf3ca5 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:57.209428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:57.218139 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf62ab-5089-4706-b084-5506eb5de507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:57.218751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf62ab-5089-4706-b084-5506eb5de507 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:57.219026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf62ab-5089-4706-b084-5506eb5de507 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:57.219295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf62ab-5089-4706-b084-5506eb5de507 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:57.220384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf62ab-5089-4706-b084-5506eb5de507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:57.221797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf62ab-5089-4706-b084-5506eb5de507 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:57.223396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 567/1137] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:12:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:12:57.229082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de094bf-d2b0-40fa-8d07-13fa4f7e2431 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:12:57.328039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de094bf-d2b0-40fa-8d07-13fa4f7e2431 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:12:57.362263 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f223d-e202-4031-aa09-f52b5242b596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:12:57.362858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f223d-e202-4031-aa09-f52b5242b596 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:57.363135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f223d-e202-4031-aa09-f52b5242b596 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:57.363403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f223d-e202-4031-aa09-f52b5242b596 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:57.364040 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f223d-e202-4031-aa09-f52b5242b596 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:12:57.365458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f223d-e202-4031-aa09-f52b5242b596 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:12:57.386317 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 571/1138] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:12:57 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:12:59.549753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efcea9-b9c6-4740-8e3a-f84a80117e72 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:59.550340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efcea9-b9c6-4740-8e3a-f84a80117e72 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:59.550340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efcea9-b9c6-4740-8e3a-f84a80117e72 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:59.551660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 568/1139] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:12:59.557024 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4872e17-d56c-46e3-87ab-a2770e7557f0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:12:59.557429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4872e17-d56c-46e3-87ab-a2770e7557f0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:12:59.557571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4872e17-d56c-46e3-87ab-a2770e7557f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:12:59.575985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a4872e17-d56c-46e3-87ab-a2770e7557f0 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:12:59.577106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a4872e17-d56c-46e3-87ab-a2770e7557f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:12:59.621100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:12:59.642079 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 572/1140] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:12:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:12:59.909560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efcea9-b9c6-4740-8e3a-f84a80117e72 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:00.009248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efcea9-b9c6-4740-8e3a-f84a80117e72 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:00.041702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe4327-58d6-4a19-9dc4-2361d2b5c7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:00.042338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe4327-58d6-4a19-9dc4-2361d2b5c7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:00.042618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe4327-58d6-4a19-9dc4-2361d2b5c7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:00.042891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe4327-58d6-4a19-9dc4-2361d2b5c7b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:00.053286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe4327-58d6-4a19-9dc4-2361d2b5c7b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ccfe4327-58d6-4a19-9dc4-2361d2b5c7b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:13:00.057981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:13:00.059649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:13:00.060852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:13:00.060852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:13:00.060852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:13:00.060852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:13:00.060852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:13:00.060852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:13:00.060852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:13:00.060852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:13:00.060852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:13:00.060852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:13:00.060852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:13:00.060852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:13:00.060852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 569/1141] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:12:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:13:00.065582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4872e17-d56c-46e3-87ab-a2770e7557f0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:00.160366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4872e17-d56c-46e3-87ab-a2770e7557f0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:00.192899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9298957-05dc-44d2-ab1b-b26eee2a89d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:00.193482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9298957-05dc-44d2-ab1b-b26eee2a89d8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:00.193762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9298957-05dc-44d2-ab1b-b26eee2a89d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:00.194033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9298957-05dc-44d2-ab1b-b26eee2a89d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:00.194712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9298957-05dc-44d2-ab1b-b26eee2a89d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:13:00.195013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9298957-05dc-44d2-ab1b-b26eee2a89d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:00.196362 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9298957-05dc-44d2-ab1b-b26eee2a89d8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:00.204579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 573/1142] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:13:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:00.210309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe4327-58d6-4a19-9dc4-2361d2b5c7b6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:00.267876 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:00.311103 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe4327-58d6-4a19-9dc4-2361d2b5c7b6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:00.344736 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc57009-0031-44c8-ab3c-9939472c984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:00.345354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc57009-0031-44c8-ab3c-9939472c984f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:00.345632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc57009-0031-44c8-ab3c-9939472c984f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:00.345898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc57009-0031-44c8-ab3c-9939472c984f None admin] PATH_INFO: `/v3/users/8fc05ac05a854950a697e8808a0413a5` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:00.359620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc57009-0031-44c8-ab3c-9939472c984f None admin] RBAC: Authorizing `identity:get_user(user_id=8fc05ac05a854950a697e8808a0413a5)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:00.361049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc57009-0031-44c8-ab3c-9939472c984f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:00.362681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 570/1143] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:13:00 2022] GET /identity/v3/users/8fc05ac05a854950a697e8808a0413a5 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:00.368482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9298957-05dc-44d2-ab1b-b26eee2a89d8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:00.411030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:00.465061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9298957-05dc-44d2-ab1b-b26eee2a89d8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:00.497772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17de96d-66f1-4b94-a9ad-2a44e8b91804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:00.498388 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17de96d-66f1-4b94-a9ad-2a44e8b91804 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:00.498667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17de96d-66f1-4b94-a9ad-2a44e8b91804 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:00.498935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17de96d-66f1-4b94-a9ad-2a44e8b91804 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:00.562886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17de96d-66f1-4b94-a9ad-2a44e8b91804 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:00.564314 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17de96d-66f1-4b94-a9ad-2a44e8b91804 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:00.606218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:00.660941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 574/1144] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:13:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:13:00.666946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc57009-0031-44c8-ab3c-9939472c984f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:00.761857 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc57009-0031-44c8-ab3c-9939472c984f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:00.774812 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:00.795136 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80ef9c-4189-4f1b-82aa-13df418a2808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:00.795748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80ef9c-4189-4f1b-82aa-13df418a2808 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:00.796020 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80ef9c-4189-4f1b-82aa-13df418a2808 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:00.796284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80ef9c-4189-4f1b-82aa-13df418a2808 None admin] PATH_INFO: `/v3/projects/de45b2f100ff449ca1fbd5bac6285a7d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:00.797376 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80ef9c-4189-4f1b-82aa-13df418a2808 None admin] RBAC: Authorizing `identity:get_project(project_id=de45b2f100ff449ca1fbd5bac6285a7d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:00.798824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80ef9c-4189-4f1b-82aa-13df418a2808 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:00.800417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 571/1145] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:13:00 2022] GET /identity/v3/projects/de45b2f100ff449ca1fbd5bac6285a7d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:00.806451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17de96d-66f1-4b94-a9ad-2a44e8b91804 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:00.907230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17de96d-66f1-4b94-a9ad-2a44e8b91804 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:00.940251 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513af97-11ba-43f4-a3ea-ff3a4280ec90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:00.940809 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513af97-11ba-43f4-a3ea-ff3a4280ec90 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:00.941083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513af97-11ba-43f4-a3ea-ff3a4280ec90 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:00.941351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513af97-11ba-43f4-a3ea-ff3a4280ec90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:00.942093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513af97-11ba-43f4-a3ea-ff3a4280ec90 None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=de45b2f100ff449ca1fbd5bac6285a7d, user.id=8fc05ac05a854950a697e8808a0413a5) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:13:00.942401 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513af97-11ba-43f4-a3ea-ff3a4280ec90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:00.943809 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513af97-11ba-43f4-a3ea-ff3a4280ec90 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:00.948950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 575/1146] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:13:00 2022] GET /identity/v3/role_assignments?user.id=8fc05ac05a854950a697e8808a0413a5&scope.project.id=de45b2f100ff449ca1fbd5bac6285a7d&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:03.109474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea3a27-6cde-441c-b7f0-e8f5b3507a76 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:03.110004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea3a27-6cde-441c-b7f0-e8f5b3507a76 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:03.110004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea3a27-6cde-441c-b7f0-e8f5b3507a76 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:03.111364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 572/1147] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:13:03.116547 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab9d58-a49f-4b12-bd3d-7c789bc16fc4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:03.116946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab9d58-a49f-4b12-bd3d-7c789bc16fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:03.117089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab9d58-a49f-4b12-bd3d-7c789bc16fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:03.130203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:03.135593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-26ab9d58-a49f-4b12-bd3d-7c789bc16fc4 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:13:03.136777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-26ab9d58-a49f-4b12-bd3d-7c789bc16fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:13:03.204275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 576/1148] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:13:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:13:03.472167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea3a27-6cde-441c-b7f0-e8f5b3507a76 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:03.570059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea3a27-6cde-441c-b7f0-e8f5b3507a76 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:03.602031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025f1b2-da6a-49ea-b205-7da10f65ab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:03.602661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025f1b2-da6a-49ea-b205-7da10f65ab0f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:03.602932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025f1b2-da6a-49ea-b205-7da10f65ab0f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:03.603193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025f1b2-da6a-49ea-b205-7da10f65ab0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:03.613624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025f1b2-da6a-49ea-b205-7da10f65ab0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-5025f1b2-da6a-49ea-b205-7da10f65ab0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:13:03.618295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:13:03.619405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:13:03.620274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:13:03.620274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:13:03.620274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:13:03.620274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:13:03.620274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:13:03.620274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:13:03.620274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:13:03.620274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:13:03.620274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:13:03.620274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:13:03.620274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:13:03.620274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:13:03.620274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 573/1149] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:13:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:13:03.625286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab9d58-a49f-4b12-bd3d-7c789bc16fc4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:03.719227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab9d58-a49f-4b12-bd3d-7c789bc16fc4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:03.751781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483b5d7-4da7-4579-8c92-fce7776ae984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:03.752305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483b5d7-4da7-4579-8c92-fce7776ae984 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:03.752579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483b5d7-4da7-4579-8c92-fce7776ae984 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:03.752847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483b5d7-4da7-4579-8c92-fce7776ae984 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:03.753494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483b5d7-4da7-4579-8c92-fce7776ae984 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:13:03.753788 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483b5d7-4da7-4579-8c92-fce7776ae984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:03.755162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483b5d7-4da7-4579-8c92-fce7776ae984 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:03.757218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:03.764054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 577/1150] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:13:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:03.769662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025f1b2-da6a-49ea-b205-7da10f65ab0f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:03.811194 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:03.868197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025f1b2-da6a-49ea-b205-7da10f65ab0f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:03.900932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bec48c-3032-4635-aeac-74083d6640ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:03.901539 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bec48c-3032-4635-aeac-74083d6640ce None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:03.901819 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bec48c-3032-4635-aeac-74083d6640ce None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:03.902109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bec48c-3032-4635-aeac-74083d6640ce None admin] PATH_INFO: `/v3/users/8fc05ac05a854950a697e8808a0413a5` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:03.903739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bec48c-3032-4635-aeac-74083d6640ce None admin] RBAC: Authorizing `identity:get_user(user_id=8fc05ac05a854950a697e8808a0413a5)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:03.905229 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bec48c-3032-4635-aeac-74083d6640ce None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:03.906864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 574/1151] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:13:03 2022] GET /identity/v3/users/8fc05ac05a854950a697e8808a0413a5 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:03.912720 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483b5d7-4da7-4579-8c92-fce7776ae984 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:04.008731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483b5d7-4da7-4579-8c92-fce7776ae984 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:04.041272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91161b0-56c2-4cff-ad81-c96364bffc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:04.041875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91161b0-56c2-4cff-ad81-c96364bffc59 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:04.042179 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91161b0-56c2-4cff-ad81-c96364bffc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:04.042452 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91161b0-56c2-4cff-ad81-c96364bffc59 None admin] PATH_INFO: `/v3/projects/de45b2f100ff449ca1fbd5bac6285a7d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:04.043577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91161b0-56c2-4cff-ad81-c96364bffc59 None admin] RBAC: Authorizing `identity:get_project(project_id=de45b2f100ff449ca1fbd5bac6285a7d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:04.044992 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91161b0-56c2-4cff-ad81-c96364bffc59 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:04.046632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 578/1152] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:13:03 2022] GET /identity/v3/projects/de45b2f100ff449ca1fbd5bac6285a7d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:04.052408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bec48c-3032-4635-aeac-74083d6640ce None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:04.148140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bec48c-3032-4635-aeac-74083d6640ce None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:04.157091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:04.180869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8714b-a7a6-4646-95d9-c4fefed1f620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:04.181512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8714b-a7a6-4646-95d9-c4fefed1f620 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:04.181788 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8714b-a7a6-4646-95d9-c4fefed1f620 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:04.182061 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8714b-a7a6-4646-95d9-c4fefed1f620 None admin] PATH_INFO: `/v3/projects/de45b2f100ff449ca1fbd5bac6285a7d/users/8fc05ac05a854950a697e8808a0413a5/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:04.183984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8714b-a7a6-4646-95d9-c4fefed1f620 None admin] RBAC: Authorizing `identity:create_grant(project_id=de45b2f100ff449ca1fbd5bac6285a7d, user_id=8fc05ac05a854950a697e8808a0413a5, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:04.185400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8714b-a7a6-4646-95d9-c4fefed1f620 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:04.198055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 575/1153] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:13:04 2022] PUT /identity/v3/projects/de45b2f100ff449ca1fbd5bac6285a7d/users/8fc05ac05a854950a697e8808a0413a5/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:13:06.291991 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790a53a-53fa-4aae-9433-cb0d83382da6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:06.292642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790a53a-53fa-4aae-9433-cb0d83382da6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:06.292642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e790a53a-53fa-4aae-9433-cb0d83382da6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:06.294365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 579/1154] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:13:06.299507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e16e7c-656c-4c21-9ece-abbe7cb37e03 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:06.299910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e16e7c-656c-4c21-9ece-abbe7cb37e03 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:06.300046 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e16e7c-656c-4c21-9ece-abbe7cb37e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:06.318816 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f2e16e7c-656c-4c21-9ece-abbe7cb37e03 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:13:06.319980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f2e16e7c-656c-4c21-9ece-abbe7cb37e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:13:06.383080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 576/1155] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:13:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:13:06.652719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790a53a-53fa-4aae-9433-cb0d83382da6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:06.717710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:06.755389 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e790a53a-53fa-4aae-9433-cb0d83382da6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:06.787034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a985ca2-01a3-435d-97ae-04b03b8d12a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:06.787639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a985ca2-01a3-435d-97ae-04b03b8d12a7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:06.787911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a985ca2-01a3-435d-97ae-04b03b8d12a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:06.788176 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a985ca2-01a3-435d-97ae-04b03b8d12a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:06.798207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a985ca2-01a3-435d-97ae-04b03b8d12a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-3a985ca2-01a3-435d-97ae-04b03b8d12a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:13:06.802813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:13:06.804039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:13:06.805207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:13:06.805207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:13:06.805207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:13:06.805207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:13:06.805207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:13:06.805207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:13:06.805207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:13:06.805207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:13:06.805207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:13:06.805207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:13:06.805207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:13:06.805207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:13:06.805207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 580/1156] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:13:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:13:06.810589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e16e7c-656c-4c21-9ece-abbe7cb37e03 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:06.901595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:06.905611 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e16e7c-656c-4c21-9ece-abbe7cb37e03 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:06.939416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af41cc-89b9-4c8c-8c95-4920d184f6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:06.939998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af41cc-89b9-4c8c-8c95-4920d184f6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:06.940269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af41cc-89b9-4c8c-8c95-4920d184f6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:06.940535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af41cc-89b9-4c8c-8c95-4920d184f6a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:06.941184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af41cc-89b9-4c8c-8c95-4920d184f6a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:13:06.941477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af41cc-89b9-4c8c-8c95-4920d184f6a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:06.942851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af41cc-89b9-4c8c-8c95-4920d184f6a3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:06.951131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 577/1157] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:13:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:06.957856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a985ca2-01a3-435d-97ae-04b03b8d12a7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:07.053983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a985ca2-01a3-435d-97ae-04b03b8d12a7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:07.078217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:07.086970 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0954779-8b92-4418-9abc-0d7b4c92d36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:07.087579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0954779-8b92-4418-9abc-0d7b4c92d36a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:07.087856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0954779-8b92-4418-9abc-0d7b4c92d36a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:07.088126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0954779-8b92-4418-9abc-0d7b4c92d36a None admin] PATH_INFO: `/v3/users/8fc05ac05a854950a697e8808a0413a5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:07.089228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0954779-8b92-4418-9abc-0d7b4c92d36a None admin] RBAC: Authorizing `identity:get_user(user_id=8fc05ac05a854950a697e8808a0413a5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:07.090707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0954779-8b92-4418-9abc-0d7b4c92d36a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:07.092253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 581/1158] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:13:06 2022] GET /identity/v3/users/8fc05ac05a854950a697e8808a0413a5 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:07.097883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af41cc-89b9-4c8c-8c95-4920d184f6a3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:07.194364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af41cc-89b9-4c8c-8c95-4920d184f6a3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:07.227096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da0fcf-b370-4d9a-a7da-44da9e6acdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:07.227680 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da0fcf-b370-4d9a-a7da-44da9e6acdbf None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:07.227955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da0fcf-b370-4d9a-a7da-44da9e6acdbf None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:07.228221 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da0fcf-b370-4d9a-a7da-44da9e6acdbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:07.254498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:07.294157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da0fcf-b370-4d9a-a7da-44da9e6acdbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:07.295580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da0fcf-b370-4d9a-a7da-44da9e6acdbf None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:07.391766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 578/1159] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:13:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:13:07.397590 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0954779-8b92-4418-9abc-0d7b4c92d36a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:07.493945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0954779-8b92-4418-9abc-0d7b4c92d36a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:07.526364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280288da-d0eb-45fd-8f96-d533ab4bb527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:07.526969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280288da-d0eb-45fd-8f96-d533ab4bb527 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:07.527247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280288da-d0eb-45fd-8f96-d533ab4bb527 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:07.527518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280288da-d0eb-45fd-8f96-d533ab4bb527 None admin] PATH_INFO: `/v3/projects/de45b2f100ff449ca1fbd5bac6285a7d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:07.528613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280288da-d0eb-45fd-8f96-d533ab4bb527 None admin] RBAC: Authorizing `identity:get_project(project_id=de45b2f100ff449ca1fbd5bac6285a7d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:07.530024 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280288da-d0eb-45fd-8f96-d533ab4bb527 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:07.531655 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 582/1160] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:13:07 2022] GET /identity/v3/projects/de45b2f100ff449ca1fbd5bac6285a7d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:07.537595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da0fcf-b370-4d9a-a7da-44da9e6acdbf None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:07.594178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:07.633702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da0fcf-b370-4d9a-a7da-44da9e6acdbf None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:07.666632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16749f43-1c80-4621-a2ad-9503000e759a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:07.667221 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16749f43-1c80-4621-a2ad-9503000e759a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:07.667496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16749f43-1c80-4621-a2ad-9503000e759a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:07.667765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16749f43-1c80-4621-a2ad-9503000e759a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:07.669016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16749f43-1c80-4621-a2ad-9503000e759a None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=de45b2f100ff449ca1fbd5bac6285a7d, user.id=8fc05ac05a854950a697e8808a0413a5) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:13:07.669320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16749f43-1c80-4621-a2ad-9503000e759a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:07.670763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16749f43-1c80-4621-a2ad-9503000e759a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:07.676551 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 579/1161] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:13:07 2022] GET /identity/v3/role_assignments?user.id=8fc05ac05a854950a697e8808a0413a5&scope.project.id=de45b2f100ff449ca1fbd5bac6285a7d&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:09.821445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a884b5-d0bb-41da-a876-9c7219643b67 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:09.822002 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a884b5-d0bb-41da-a876-9c7219643b67 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:09.822002 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a884b5-d0bb-41da-a876-9c7219643b67 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:09.823387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 583/1162] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:13:09.828696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab766e4-3911-4e13-8b9d-a4e2ca9abd77 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:09.829089 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab766e4-3911-4e13-8b9d-a4e2ca9abd77 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:09.829233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab766e4-3911-4e13-8b9d-a4e2ca9abd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:09.847535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5ab766e4-3911-4e13-8b9d-a4e2ca9abd77 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:13:09.848701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5ab766e4-3911-4e13-8b9d-a4e2ca9abd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:13:09.912288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 580/1163] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:13:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:13:10.180283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a884b5-d0bb-41da-a876-9c7219643b67 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:10.241980 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:10.280772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a884b5-d0bb-41da-a876-9c7219643b67 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:10.312872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399c122-8504-446d-81e9-8c5dd151547e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:10.313469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399c122-8504-446d-81e9-8c5dd151547e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:10.313745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399c122-8504-446d-81e9-8c5dd151547e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:10.314016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399c122-8504-446d-81e9-8c5dd151547e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:10.378769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399c122-8504-446d-81e9-8c5dd151547e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:10.380195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399c122-8504-446d-81e9-8c5dd151547e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:10.437818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:10.480456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 584/1164] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:13:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:13:10.486981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab766e4-3911-4e13-8b9d-a4e2ca9abd77 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:10.540601 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:10.583846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab766e4-3911-4e13-8b9d-a4e2ca9abd77 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:10.617733 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff135f3d-edf1-4a7e-b505-bba9350c44fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:10.618387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff135f3d-edf1-4a7e-b505-bba9350c44fa None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:10.618663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff135f3d-edf1-4a7e-b505-bba9350c44fa None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:10.618930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff135f3d-edf1-4a7e-b505-bba9350c44fa None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:10.625992 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff135f3d-edf1-4a7e-b505-bba9350c44fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:10.627446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff135f3d-edf1-4a7e-b505-bba9350c44fa None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ff135f3d-edf1-4a7e-b505-bba9350c44fa None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:13:10.639097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:13:10.640329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:13:10.641200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 581/1165] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Sep 27 17:13:10 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:13:10.646299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399c122-8504-446d-81e9-8c5dd151547e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:10.740465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399c122-8504-446d-81e9-8c5dd151547e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:10.778677 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f6331b-90a3-4aa9-bdb4-07d667eedf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:10.779276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6331b-90a3-4aa9-bdb4-07d667eedf40 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:10.779552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6331b-90a3-4aa9-bdb4-07d667eedf40 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:10.779822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f6331b-90a3-4aa9-bdb4-07d667eedf40 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:10.780460 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6331b-90a3-4aa9-bdb4-07d667eedf40 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:13:10.780759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6331b-90a3-4aa9-bdb4-07d667eedf40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:10.782137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f6331b-90a3-4aa9-bdb4-07d667eedf40 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:10.789161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 585/1166] 10.222.0.55 () {56 vars in 1115 bytes} [Tue Sep 27 17:13:10 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:12.928913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec40f63-5081-400e-b642-6f018f43276c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:12.929477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec40f63-5081-400e-b642-6f018f43276c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:12.929477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec40f63-5081-400e-b642-6f018f43276c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:12.930948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 582/1167] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:13:12.936328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7f1f3-4380-4ad5-8659-2c63830691dd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:12.936769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7f1f3-4380-4ad5-8659-2c63830691dd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:12.936872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7f1f3-4380-4ad5-8659-2c63830691dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:12.955152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a6f7f1f3-4380-4ad5-8659-2c63830691dd None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:13:12.956309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a6f7f1f3-4380-4ad5-8659-2c63830691dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:13:12.975909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:13.020372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 586/1168] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:13:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:13:13.290389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec40f63-5081-400e-b642-6f018f43276c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:13.387450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec40f63-5081-400e-b642-6f018f43276c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:13.400475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:13.420333 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22bdddc-63a8-4211-8cb8-be8d2022308b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:13.420946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bdddc-63a8-4211-8cb8-be8d2022308b None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:13.421217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bdddc-63a8-4211-8cb8-be8d2022308b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:13.421480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22bdddc-63a8-4211-8cb8-be8d2022308b None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:13.422166 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22bdddc-63a8-4211-8cb8-be8d2022308b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:13.423516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22bdddc-63a8-4211-8cb8-be8d2022308b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:13.439612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 583/1169] 10.222.0.55 () {60 vars in 1138 bytes} [Tue Sep 27 17:13:13 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:13:15.605520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7f3c0-b2bc-471f-a153-ba8c86968278 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:15.606045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7f3c0-b2bc-471f-a153-ba8c86968278 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:15.606045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7f3c0-b2bc-471f-a153-ba8c86968278 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:15.607420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 587/1170] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:13:15.612570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcec61b-a569-45b3-b89f-01676359197a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:15.612962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcec61b-a569-45b3-b89f-01676359197a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:15.613116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcec61b-a569-45b3-b89f-01676359197a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:15.631268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-dfcec61b-a569-45b3-b89f-01676359197a None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:13:15.632448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-dfcec61b-a569-45b3-b89f-01676359197a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:13:15.696080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 584/1171] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:13:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:13:15.966102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7f3c0-b2bc-471f-a153-ba8c86968278 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:16.062990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7f3c0-b2bc-471f-a153-ba8c86968278 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:16.098904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed63f79-f7a0-4c57-bb33-dfc21d2e77b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:16.099542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed63f79-f7a0-4c57-bb33-dfc21d2e77b1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:16.099817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed63f79-f7a0-4c57-bb33-dfc21d2e77b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:16.100086 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed63f79-f7a0-4c57-bb33-dfc21d2e77b1 None admin] PATH_INFO: `/v3/domains/3d54453699d742a993ce4fe7f26197ae` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:16.110862 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed63f79-f7a0-4c57-bb33-dfc21d2e77b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3d54453699d742a993ce4fe7f26197ae)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:16.112281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed63f79-f7a0-4c57-bb33-dfc21d2e77b1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:16.113884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 588/1172] 10.222.0.55 () {56 vars in 1183 bytes} [Tue Sep 27 17:13:15 2022] GET /identity/v3/domains/3d54453699d742a993ce4fe7f26197ae => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:16.119762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcec61b-a569-45b3-b89f-01676359197a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:16.214243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcec61b-a569-45b3-b89f-01676359197a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:16.217138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:16.247802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d11fe-f02c-4ddf-9363-c7e3197478a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:16.248400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d11fe-f02c-4ddf-9363-c7e3197478a8 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:16.248679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d11fe-f02c-4ddf-9363-c7e3197478a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:16.248952 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d11fe-f02c-4ddf-9363-c7e3197478a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:16.249588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d11fe-f02c-4ddf-9363-c7e3197478a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:16.251033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d11fe-f02c-4ddf-9363-c7e3197478a8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:16.276244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 585/1173] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:13:16 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:13:18.472663 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8b56c-4cac-4e57-b332-9dd98032979d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:18.473202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8b56c-4cac-4e57-b332-9dd98032979d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:18.473307 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8b56c-4cac-4e57-b332-9dd98032979d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:18.474606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 589/1174] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:13:18.480035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6df2e7-cc6b-4a0f-b208-cb95c35b9a4f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:18.480417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6df2e7-cc6b-4a0f-b208-cb95c35b9a4f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:18.480596 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6df2e7-cc6b-4a0f-b208-cb95c35b9a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:18.498495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0c6df2e7-cc6b-4a0f-b208-cb95c35b9a4f None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:13:18.499661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0c6df2e7-cc6b-4a0f-b208-cb95c35b9a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:13:18.563786 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 586/1175] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:13:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:13:18.831345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8b56c-4cac-4e57-b332-9dd98032979d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:18.849899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:18.931703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8b56c-4cac-4e57-b332-9dd98032979d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:18.965191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d98fe-f0be-48fc-965a-efd6c5062b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:18.965799 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d98fe-f0be-48fc-965a-efd6c5062b66 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:18.966100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d98fe-f0be-48fc-965a-efd6c5062b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:18.966363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d98fe-f0be-48fc-965a-efd6c5062b66 None admin] PATH_INFO: `/v3/domains/3d54453699d742a993ce4fe7f26197ae` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:18.967459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d98fe-f0be-48fc-965a-efd6c5062b66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3d54453699d742a993ce4fe7f26197ae)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:18.968875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d98fe-f0be-48fc-965a-efd6c5062b66 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:18.970469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 590/1176] 10.222.0.55 () {56 vars in 1183 bytes} [Tue Sep 27 17:13:18 2022] GET /identity/v3/domains/3d54453699d742a993ce4fe7f26197ae => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:18.976256 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6df2e7-cc6b-4a0f-b208-cb95c35b9a4f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:19.056715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:19.072532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6df2e7-cc6b-4a0f-b208-cb95c35b9a4f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:19.106735 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1de8e7-f3db-4e80-aca7-6f896a504399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:19.107320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1de8e7-f3db-4e80-aca7-6f896a504399 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:19.107593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1de8e7-f3db-4e80-aca7-6f896a504399 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:19.107860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1de8e7-f3db-4e80-aca7-6f896a504399 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:19.108490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1de8e7-f3db-4e80-aca7-6f896a504399 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:19.109902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1de8e7-f3db-4e80-aca7-6f896a504399 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:19.129610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 587/1177] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:13:18 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:13:21.297283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d333d4d-a770-4e64-b1cc-89b2a14eb7eb None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:21.297813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d333d4d-a770-4e64-b1cc-89b2a14eb7eb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:21.297813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d333d4d-a770-4e64-b1cc-89b2a14eb7eb None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:21.299205 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 591/1178] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:13:21.304340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66783752-a94d-4771-966d-918b659afbbb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:21.304763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66783752-a94d-4771-966d-918b659afbbb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:21.304874 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66783752-a94d-4771-966d-918b659afbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:21.323199 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-66783752-a94d-4771-966d-918b659afbbb None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:13:21.324365 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-66783752-a94d-4771-966d-918b659afbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:13:21.387800 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 588/1179] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:13:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:13:21.658467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d333d4d-a770-4e64-b1cc-89b2a14eb7eb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:21.740647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:21.756513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d333d4d-a770-4e64-b1cc-89b2a14eb7eb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:21.791362 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df405805-cec4-495b-84e7-78f081b21b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:21.791971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df405805-cec4-495b-84e7-78f081b21b31 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:21.792245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df405805-cec4-495b-84e7-78f081b21b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:21.793084 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df405805-cec4-495b-84e7-78f081b21b31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:21.803342 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df405805-cec4-495b-84e7-78f081b21b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-df405805-cec4-495b-84e7-78f081b21b31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:13:21.808131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:13:21.809230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:13:21.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:13:21.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:13:21.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:13:21.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:13:21.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:13:21.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:13:21.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:13:21.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:13:21.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:13:21.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:13:21.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:13:21.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:13:21.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 592/1180] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:13:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:13:21.815265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66783752-a94d-4771-966d-918b659afbbb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:21.882325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:21.911824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66783752-a94d-4771-966d-918b659afbbb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:21.943438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38879a-9deb-4abe-9693-2ce70e5f653d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:21.944026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38879a-9deb-4abe-9693-2ce70e5f653d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:21.944300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38879a-9deb-4abe-9693-2ce70e5f653d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:21.944567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38879a-9deb-4abe-9693-2ce70e5f653d None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:21.945218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38879a-9deb-4abe-9693-2ce70e5f653d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:13:21.945513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38879a-9deb-4abe-9693-2ce70e5f653d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:21.946887 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38879a-9deb-4abe-9693-2ce70e5f653d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:21.955837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 589/1181] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:13:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:21.961708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df405805-cec4-495b-84e7-78f081b21b31 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:22.062221 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df405805-cec4-495b-84e7-78f081b21b31 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:22.094626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d799fa-faf5-48b7-9032-cbc6c165d512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:22.095240 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d799fa-faf5-48b7-9032-cbc6c165d512 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:22.095518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d799fa-faf5-48b7-9032-cbc6c165d512 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:22.095789 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d799fa-faf5-48b7-9032-cbc6c165d512 None admin] PATH_INFO: `/v3/users/78d010f3a5194c5fb28413a9bc9acb11` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:22.109131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d799fa-faf5-48b7-9032-cbc6c165d512 None admin] RBAC: Authorizing `identity:get_user(user_id=78d010f3a5194c5fb28413a9bc9acb11)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:22.110586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d799fa-faf5-48b7-9032-cbc6c165d512 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:22.112175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 593/1182] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:13:21 2022] GET /identity/v3/users/78d010f3a5194c5fb28413a9bc9acb11 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:22.118269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38879a-9deb-4abe-9693-2ce70e5f653d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:22.214387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38879a-9deb-4abe-9693-2ce70e5f653d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:22.247343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17089d-c6f7-42dc-b2be-4fd05bf35a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:22.247937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17089d-c6f7-42dc-b2be-4fd05bf35a7b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:22.248215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17089d-c6f7-42dc-b2be-4fd05bf35a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:22.248482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17089d-c6f7-42dc-b2be-4fd05bf35a7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:22.250999 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:22.317226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17089d-c6f7-42dc-b2be-4fd05bf35a7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:22.318715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17089d-c6f7-42dc-b2be-4fd05bf35a7b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:22.412452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 590/1183] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:13:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:13:22.419020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d799fa-faf5-48b7-9032-cbc6c165d512 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:22.421827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:22.514964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d799fa-faf5-48b7-9032-cbc6c165d512 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:22.547758 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e015aef-2a48-4c5b-b621-c591aef9b071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:22.548365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e015aef-2a48-4c5b-b621-c591aef9b071 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:22.548638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e015aef-2a48-4c5b-b621-c591aef9b071 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:22.548905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e015aef-2a48-4c5b-b621-c591aef9b071 None admin] PATH_INFO: `/v3/projects/4e327144be2a4a35b84ca7502927b13f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:22.549991 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e015aef-2a48-4c5b-b621-c591aef9b071 None admin] RBAC: Authorizing `identity:get_project(project_id=4e327144be2a4a35b84ca7502927b13f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:22.551420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e015aef-2a48-4c5b-b621-c591aef9b071 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:22.553015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 594/1184] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:13:22 2022] GET /identity/v3/projects/4e327144be2a4a35b84ca7502927b13f => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:22.559152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17089d-c6f7-42dc-b2be-4fd05bf35a7b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:22.591399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:22.655501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17089d-c6f7-42dc-b2be-4fd05bf35a7b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:22.688058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c106b46-c441-4652-9420-565a69f2f742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:22.688638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c106b46-c441-4652-9420-565a69f2f742 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:22.688913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c106b46-c441-4652-9420-565a69f2f742 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:22.689181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c106b46-c441-4652-9420-565a69f2f742 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:22.689893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c106b46-c441-4652-9420-565a69f2f742 None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=4e327144be2a4a35b84ca7502927b13f, user.id=78d010f3a5194c5fb28413a9bc9acb11) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:13:22.690226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c106b46-c441-4652-9420-565a69f2f742 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:22.691639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c106b46-c441-4652-9420-565a69f2f742 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:22.696746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 591/1185] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:13:22 2022] GET /identity/v3/role_assignments?user.id=78d010f3a5194c5fb28413a9bc9acb11&scope.project.id=4e327144be2a4a35b84ca7502927b13f&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:24.815051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aab9b1-7ac1-40d8-9f1f-cf446e3447dd None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:24.815585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aab9b1-7ac1-40d8-9f1f-cf446e3447dd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:24.815585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aab9b1-7ac1-40d8-9f1f-cf446e3447dd None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:24.816914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 595/1186] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:13:24.822288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bfee1-88f0-456f-a41c-f8200f6241bb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:24.822727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bfee1-88f0-456f-a41c-f8200f6241bb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:24.822822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bfee1-88f0-456f-a41c-f8200f6241bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:24.841043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e93bfee1-88f0-456f-a41c-f8200f6241bb None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:13:24.842255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e93bfee1-88f0-456f-a41c-f8200f6241bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:13:24.906248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 592/1187] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:13:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:13:25.178231 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aab9b1-7ac1-40d8-9f1f-cf446e3447dd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:25.247418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:25.280027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aab9b1-7ac1-40d8-9f1f-cf446e3447dd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:25.312581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc09c60-35d7-4b9c-bdc8-9837c0776e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:25.313199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc09c60-35d7-4b9c-bdc8-9837c0776e6d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:25.313476 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc09c60-35d7-4b9c-bdc8-9837c0776e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:25.313742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc09c60-35d7-4b9c-bdc8-9837c0776e6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:25.323645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc09c60-35d7-4b9c-bdc8-9837c0776e6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-cdc09c60-35d7-4b9c-bdc8-9837c0776e6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:13:25.328380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:13:25.329507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:13:25.330455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:13:25.330455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:13:25.330455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:13:25.330455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:13:25.330455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:13:25.330455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:13:25.330455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:13:25.330455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:13:25.330455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:13:25.330455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:13:25.330455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:13:25.330455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:13:25.330455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 596/1188] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:13:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:13:25.335483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bfee1-88f0-456f-a41c-f8200f6241bb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:25.363028 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:25.431151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bfee1-88f0-456f-a41c-f8200f6241bb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:25.462809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019bdce-cf20-4b7d-90ed-9c78ca5dd0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:25.463399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019bdce-cf20-4b7d-90ed-9c78ca5dd0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:25.463671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019bdce-cf20-4b7d-90ed-9c78ca5dd0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:25.463934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019bdce-cf20-4b7d-90ed-9c78ca5dd0f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:25.465108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019bdce-cf20-4b7d-90ed-9c78ca5dd0f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:13:25.465407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019bdce-cf20-4b7d-90ed-9c78ca5dd0f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:25.466791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019bdce-cf20-4b7d-90ed-9c78ca5dd0f7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:25.474441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 593/1189] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:13:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:25.480398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc09c60-35d7-4b9c-bdc8-9837c0776e6d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:25.576142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc09c60-35d7-4b9c-bdc8-9837c0776e6d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:25.603300 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:25.609884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644ff9b-fd9a-40bf-87d6-dd99579bb77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:25.610516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644ff9b-fd9a-40bf-87d6-dd99579bb77b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:25.610791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644ff9b-fd9a-40bf-87d6-dd99579bb77b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:25.611061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644ff9b-fd9a-40bf-87d6-dd99579bb77b None admin] PATH_INFO: `/v3/users/78d010f3a5194c5fb28413a9bc9acb11` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:25.612159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644ff9b-fd9a-40bf-87d6-dd99579bb77b None admin] RBAC: Authorizing `identity:get_user(user_id=78d010f3a5194c5fb28413a9bc9acb11)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:25.613565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644ff9b-fd9a-40bf-87d6-dd99579bb77b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:25.615143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 597/1190] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:13:25 2022] GET /identity/v3/users/78d010f3a5194c5fb28413a9bc9acb11 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:25.620888 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019bdce-cf20-4b7d-90ed-9c78ca5dd0f7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:25.707554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:25.717764 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019bdce-cf20-4b7d-90ed-9c78ca5dd0f7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:25.749343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b93a7-9eac-4bde-aaf9-30e9147215e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:25.749960 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b93a7-9eac-4bde-aaf9-30e9147215e5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:25.750273 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b93a7-9eac-4bde-aaf9-30e9147215e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:25.750546 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b93a7-9eac-4bde-aaf9-30e9147215e5 None admin] PATH_INFO: `/v3/projects/4e327144be2a4a35b84ca7502927b13f` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:25.751647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b93a7-9eac-4bde-aaf9-30e9147215e5 None admin] RBAC: Authorizing `identity:get_project(project_id=4e327144be2a4a35b84ca7502927b13f)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:25.753069 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8b93a7-9eac-4bde-aaf9-30e9147215e5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:25.754714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 594/1191] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:13:25 2022] GET /identity/v3/projects/4e327144be2a4a35b84ca7502927b13f => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:25.760473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644ff9b-fd9a-40bf-87d6-dd99579bb77b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:25.866016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644ff9b-fd9a-40bf-87d6-dd99579bb77b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:25.898771 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa1416-0f43-40b1-9be8-5bf38d700852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:25.899388 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1416-0f43-40b1-9be8-5bf38d700852 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:25.899661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1416-0f43-40b1-9be8-5bf38d700852 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:25.899929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1416-0f43-40b1-9be8-5bf38d700852 None admin] PATH_INFO: `/v3/projects/4e327144be2a4a35b84ca7502927b13f/users/78d010f3a5194c5fb28413a9bc9acb11/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:25.902407 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa1416-0f43-40b1-9be8-5bf38d700852 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e327144be2a4a35b84ca7502927b13f, user_id=78d010f3a5194c5fb28413a9bc9acb11, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:25.903821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa1416-0f43-40b1-9be8-5bf38d700852 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:25.915954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 598/1192] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:13:25 2022] PUT /identity/v3/projects/4e327144be2a4a35b84ca7502927b13f/users/78d010f3a5194c5fb28413a9bc9acb11/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:13:28.044903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b271244-d82a-4edb-9f82-0aabe3606148 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:28.045451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b271244-d82a-4edb-9f82-0aabe3606148 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:28.045451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b271244-d82a-4edb-9f82-0aabe3606148 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:28.046825 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 595/1193] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:13:28.052238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39e947-9905-436e-ac63-09d2279e2ef4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:28.052629 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39e947-9905-436e-ac63-09d2279e2ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:28.052787 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39e947-9905-436e-ac63-09d2279e2ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:28.071153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5e39e947-9905-436e-ac63-09d2279e2ef4 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:13:28.072337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5e39e947-9905-436e-ac63-09d2279e2ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:13:28.123335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:28.137203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 599/1194] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:13:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:13:28.407108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b271244-d82a-4edb-9f82-0aabe3606148 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:28.504458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b271244-d82a-4edb-9f82-0aabe3606148 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:28.536485 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68b02b-55b1-42b6-8ded-1f5f2c2f5561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:28.537086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68b02b-55b1-42b6-8ded-1f5f2c2f5561 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:28.537358 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68b02b-55b1-42b6-8ded-1f5f2c2f5561 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:28.537620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68b02b-55b1-42b6-8ded-1f5f2c2f5561 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:28.547779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df68b02b-55b1-42b6-8ded-1f5f2c2f5561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-df68b02b-55b1-42b6-8ded-1f5f2c2f5561 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:13:28.552420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:13:28.553511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:13:28.554443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:13:28.554443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:13:28.554443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:13:28.554443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:13:28.554443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:13:28.554443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:13:28.554443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:13:28.554443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:13:28.554443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:13:28.554443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:13:28.554443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:13:28.554443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:13:28.554443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 596/1195] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:13:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:13:28.559430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39e947-9905-436e-ac63-09d2279e2ef4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:28.653528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39e947-9905-436e-ac63-09d2279e2ef4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:28.685195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e573958-84fb-41ca-9b2b-d38dfb7ba31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:28.685771 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e573958-84fb-41ca-9b2b-d38dfb7ba31b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:28.686042 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e573958-84fb-41ca-9b2b-d38dfb7ba31b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:28.686334 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e573958-84fb-41ca-9b2b-d38dfb7ba31b None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:28.686979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e573958-84fb-41ca-9b2b-d38dfb7ba31b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:13:28.687272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e573958-84fb-41ca-9b2b-d38dfb7ba31b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:28.688608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e573958-84fb-41ca-9b2b-d38dfb7ba31b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:28.696257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 600/1196] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:13:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:28.702421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68b02b-55b1-42b6-8ded-1f5f2c2f5561 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:28.721721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:28.799494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68b02b-55b1-42b6-8ded-1f5f2c2f5561 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:28.832429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e7b70-171a-4059-8030-9b70e407015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:28.833036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e7b70-171a-4059-8030-9b70e407015a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:28.833314 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e7b70-171a-4059-8030-9b70e407015a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:28.833582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e7b70-171a-4059-8030-9b70e407015a None admin] PATH_INFO: `/v3/users/78d010f3a5194c5fb28413a9bc9acb11` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:28.834699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e7b70-171a-4059-8030-9b70e407015a None admin] RBAC: Authorizing `identity:get_user(user_id=78d010f3a5194c5fb28413a9bc9acb11)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:28.836118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e7b70-171a-4059-8030-9b70e407015a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:28.837656 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 597/1197] 10.222.0.55 () {56 vars in 1177 bytes} [Tue Sep 27 17:13:28 2022] GET /identity/v3/users/78d010f3a5194c5fb28413a9bc9acb11 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:28.843834 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e573958-84fb-41ca-9b2b-d38dfb7ba31b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:28.891190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:28.938236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e573958-84fb-41ca-9b2b-d38dfb7ba31b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:28.970109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827c8ba-d11b-4403-96a6-4a90ce7dea9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:28.970686 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827c8ba-d11b-4403-96a6-4a90ce7dea9e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:28.970957 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827c8ba-d11b-4403-96a6-4a90ce7dea9e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:28.971220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827c8ba-d11b-4403-96a6-4a90ce7dea9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:29.034169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827c8ba-d11b-4403-96a6-4a90ce7dea9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:29.035570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827c8ba-d11b-4403-96a6-4a90ce7dea9e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:29.084840 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:29.134373 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 601/1198] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:13:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:13:29.140900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e7b70-171a-4059-8030-9b70e407015a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:29.226746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:29.242420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e7b70-171a-4059-8030-9b70e407015a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:29.276702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53daa46-b032-400f-949c-086fbd7e45ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:29.277319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53daa46-b032-400f-949c-086fbd7e45ef None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:29.277598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53daa46-b032-400f-949c-086fbd7e45ef None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:29.277868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53daa46-b032-400f-949c-086fbd7e45ef None admin] PATH_INFO: `/v3/projects/4e327144be2a4a35b84ca7502927b13f` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:29.279078 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53daa46-b032-400f-949c-086fbd7e45ef None admin] RBAC: Authorizing `identity:get_project(project_id=4e327144be2a4a35b84ca7502927b13f)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:29.281266 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53daa46-b032-400f-949c-086fbd7e45ef None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:29.282949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 598/1199] 10.222.0.55 () {56 vars in 1186 bytes} [Tue Sep 27 17:13:29 2022] GET /identity/v3/projects/4e327144be2a4a35b84ca7502927b13f => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:29.289623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827c8ba-d11b-4403-96a6-4a90ce7dea9e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:13:29.388858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827c8ba-d11b-4403-96a6-4a90ce7dea9e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:13:29.422328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977de3fa-5ae6-4e75-b950-d8247484926d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:13:29.422914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977de3fa-5ae6-4e75-b950-d8247484926d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:13:29.423185 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977de3fa-5ae6-4e75-b950-d8247484926d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:13:29.423449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977de3fa-5ae6-4e75-b950-d8247484926d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:13:29.424165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977de3fa-5ae6-4e75-b950-d8247484926d None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=4e327144be2a4a35b84ca7502927b13f, user.id=78d010f3a5194c5fb28413a9bc9acb11) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:13:29.424465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977de3fa-5ae6-4e75-b950-d8247484926d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:13:29.425869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977de3fa-5ae6-4e75-b950-d8247484926d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:13:29.431484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 602/1200] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:13:29 2022] GET /identity/v3/role_assignments?user.id=78d010f3a5194c5fb28413a9bc9acb11&scope.project.id=4e327144be2a4a35b84ca7502927b13f&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:13:39.272521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=23 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:13:39.428941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:25.204476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94c698-2b0f-4f6b-9227-cee21f35ddcb None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:25.205084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94c698-2b0f-4f6b-9227-cee21f35ddcb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:25.205084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94c698-2b0f-4f6b-9227-cee21f35ddcb None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:25.206717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 599/1201] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:14:25.212260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feca4b1-fcfe-4026-89ea-fb7530755144 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:25.212753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feca4b1-fcfe-4026-89ea-fb7530755144 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:25.212933 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feca4b1-fcfe-4026-89ea-fb7530755144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:25.234905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2feca4b1-fcfe-4026-89ea-fb7530755144 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:14:25.236479 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2feca4b1-fcfe-4026-89ea-fb7530755144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:14:25.309068 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 603/1202] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:14:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 27 17:14:25.583332 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94c698-2b0f-4f6b-9227-cee21f35ddcb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:25.716041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94c698-2b0f-4f6b-9227-cee21f35ddcb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:25.758101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4393f-748a-4429-8654-50732fd2ab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:25.760109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4393f-748a-4429-8654-50732fd2ab6c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:25.760622 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4393f-748a-4429-8654-50732fd2ab6c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:25.761639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4393f-748a-4429-8654-50732fd2ab6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:25.776650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4393f-748a-4429-8654-50732fd2ab6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:25.778274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4393f-748a-4429-8654-50732fd2ab6c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-bcc4393f-748a-4429-8654-50732fd2ab6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:14:25.799977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:25.801101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:14:25.802236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:14:25.802236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:25.802236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:25.802236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 600/1203] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:25.807642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feca4b1-fcfe-4026-89ea-fb7530755144 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:25.924247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feca4b1-fcfe-4026-89ea-fb7530755144 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:25.954573 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:25.965961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0116d-f3c2-42a2-9fce-4acddfb6a258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:25.966624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0116d-f3c2-42a2-9fce-4acddfb6a258 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:25.966917 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0116d-f3c2-42a2-9fce-4acddfb6a258 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:25.967189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0116d-f3c2-42a2-9fce-4acddfb6a258 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:25.967985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0116d-f3c2-42a2-9fce-4acddfb6a258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:25.968579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0116d-f3c2-42a2-9fce-4acddfb6a258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:25.969488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0116d-f3c2-42a2-9fce-4acddfb6a258 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:25.990914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4393f-748a-4429-8654-50732fd2ab6c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:25.992055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 604/1204] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:26.066854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:26.144940 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4393f-748a-4429-8654-50732fd2ab6c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:26.190602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28133fbf-5561-4aea-aefc-e61bed4baae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:26.191225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28133fbf-5561-4aea-aefc-e61bed4baae0 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:26.191508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28133fbf-5561-4aea-aefc-e61bed4baae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:26.191779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28133fbf-5561-4aea-aefc-e61bed4baae0 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:26.192423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28133fbf-5561-4aea-aefc-e61bed4baae0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:26.193865 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28133fbf-5561-4aea-aefc-e61bed4baae0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:26.219275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 601/1205] 10.222.0.55 () {60 vars in 1132 bytes} [Tue Sep 27 17:14:25 2022] POST /identity/v3/users => generated 251 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:14:28.869181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b04f4-0fc0-460f-94c5-f608df5888f6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:28.869828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b04f4-0fc0-460f-94c5-f608df5888f6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:28.869828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b04f4-0fc0-460f-94c5-f608df5888f6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:28.871357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 605/1206] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:14:28.877389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314b6a5-b28e-477f-b697-28e6bfffd8e3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:28.877896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314b6a5-b28e-477f-b697-28e6bfffd8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:28.878655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314b6a5-b28e-477f-b697-28e6bfffd8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:28.899164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-4314b6a5-b28e-477f-b697-28e6bfffd8e3 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:14:28.900617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4314b6a5-b28e-477f-b697-28e6bfffd8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:14:28.981280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 602/1207] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:14:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:14:29.282313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b04f4-0fc0-460f-94c5-f608df5888f6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:29.401769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b04f4-0fc0-460f-94c5-f608df5888f6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:29.439801 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5c0c6-c501-4818-98fe-6ee7a94c77cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:29.440763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5c0c6-c501-4818-98fe-6ee7a94c77cf None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:29.441236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5c0c6-c501-4818-98fe-6ee7a94c77cf None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:29.441680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5c0c6-c501-4818-98fe-6ee7a94c77cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:29.455163 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5c0c6-c501-4818-98fe-6ee7a94c77cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-4ed5c0c6-c501-4818-98fe-6ee7a94c77cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:14:29.460262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:29.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:29.464571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:29.464571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:29.464571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:29.464571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:29.464571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:14:29.464571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:14:29.464571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:14:29.464571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:14:29.464571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:14:29.464571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:14:29.464571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:14:29.464571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:29.467665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 606/1208] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:14:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:29.473371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314b6a5-b28e-477f-b697-28e6bfffd8e3 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:29.526065 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:29.603178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314b6a5-b28e-477f-b697-28e6bfffd8e3 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:29.647633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38fd34-88a9-447a-9da9-5d9db79b88f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:29.648292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38fd34-88a9-447a-9da9-5d9db79b88f1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:29.648722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38fd34-88a9-447a-9da9-5d9db79b88f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:29.649508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38fd34-88a9-447a-9da9-5d9db79b88f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:29.650211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38fd34-88a9-447a-9da9-5d9db79b88f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:29.650512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38fd34-88a9-447a-9da9-5d9db79b88f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:29.651971 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38fd34-88a9-447a-9da9-5d9db79b88f1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:29.661653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 603/1209] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:14:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:29.668310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5c0c6-c501-4818-98fe-6ee7a94c77cf None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:29.724149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:29.785918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5c0c6-c501-4818-98fe-6ee7a94c77cf None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:29.825634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a24c4-5f4e-4e19-b077-c1060dc28e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:29.826289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a24c4-5f4e-4e19-b077-c1060dc28e63 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:29.826562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a24c4-5f4e-4e19-b077-c1060dc28e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:29.826824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a24c4-5f4e-4e19-b077-c1060dc28e63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:29.840796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a24c4-5f4e-4e19-b077-c1060dc28e63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:29.842347 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a24c4-5f4e-4e19-b077-c1060dc28e63 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-8c9a24c4-5f4e-4e19-b077-c1060dc28e63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:14:29.862190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:29.863197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:14:29.864456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:14:29.864456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:29.864456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:29.864456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 607/1210] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:29.865820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38fd34-88a9-447a-9da9-5d9db79b88f1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:30.011489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:30.015379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38fd34-88a9-447a-9da9-5d9db79b88f1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:30.054479 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a37ac3-ce4f-4393-a40c-e88958d70339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:30.055628 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a37ac3-ce4f-4393-a40c-e88958d70339 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:30.055903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a37ac3-ce4f-4393-a40c-e88958d70339 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:30.056170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a37ac3-ce4f-4393-a40c-e88958d70339 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:30.056824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a37ac3-ce4f-4393-a40c-e88958d70339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:30.057123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a37ac3-ce4f-4393-a40c-e88958d70339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:30.058549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a37ac3-ce4f-4393-a40c-e88958d70339 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:30.071929 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 604/1211] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:30.077588 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a24c4-5f4e-4e19-b077-c1060dc28e63 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:30.205918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a24c4-5f4e-4e19-b077-c1060dc28e63 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:30.207603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:30.244956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778a897-ce7b-4ef3-b888-d773996647cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:30.245638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778a897-ce7b-4ef3-b888-d773996647cc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:30.245965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778a897-ce7b-4ef3-b888-d773996647cc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:30.246299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778a897-ce7b-4ef3-b888-d773996647cc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:30.254440 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778a897-ce7b-4ef3-b888-d773996647cc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:30.256022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778a897-ce7b-4ef3-b888-d773996647cc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d778a897-ce7b-4ef3-b888-d773996647cc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:14:30.270202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:30.271653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:30.272772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:30.272772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:30.272772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:14:30.272772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:14:30.272772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:14:30.272772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:14:30.272772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:14:30.272772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:14:30.272772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 27 17:14:30.272772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:30.272772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 608/1212] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:14:30.277962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a37ac3-ce4f-4393-a40c-e88958d70339 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:30.391986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a37ac3-ce4f-4393-a40c-e88958d70339 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:30.428112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c8728-7e16-4eca-b5cc-8acd52e6d450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:30.428720 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c8728-7e16-4eca-b5cc-8acd52e6d450 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:30.428994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c8728-7e16-4eca-b5cc-8acd52e6d450 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:30.429309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c8728-7e16-4eca-b5cc-8acd52e6d450 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:30.430064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c8728-7e16-4eca-b5cc-8acd52e6d450 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:30.430386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c8728-7e16-4eca-b5cc-8acd52e6d450 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:30.431806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c8728-7e16-4eca-b5cc-8acd52e6d450 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:30.454917 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 605/1213] 10.222.0.55 () {56 vars in 1143 bytes} [Tue Sep 27 17:14:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:30.461417 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778a897-ce7b-4ef3-b888-d773996647cc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:30.585305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778a897-ce7b-4ef3-b888-d773996647cc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:30.628628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7c4f5-79d1-4621-a064-75d3c07dbea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:30.629496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7c4f5-79d1-4621-a064-75d3c07dbea5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:30.629903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7c4f5-79d1-4621-a064-75d3c07dbea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:30.630310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7c4f5-79d1-4621-a064-75d3c07dbea5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:30.669261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:30.715895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7c4f5-79d1-4621-a064-75d3c07dbea5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:30.717755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7c4f5-79d1-4621-a064-75d3c07dbea5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:30.834027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 609/1214] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:14:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 27 17:14:30.842093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c8728-7e16-4eca-b5cc-8acd52e6d450 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:30.864938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:30.965681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c8728-7e16-4eca-b5cc-8acd52e6d450 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:31.006286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713bdf2-543d-475d-a11f-286c555c22c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:31.006825 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713bdf2-543d-475d-a11f-286c555c22c0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:31.007096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713bdf2-543d-475d-a11f-286c555c22c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:31.007361 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713bdf2-543d-475d-a11f-286c555c22c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:31.017687 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713bdf2-543d-475d-a11f-286c555c22c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:31.019200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713bdf2-543d-475d-a11f-286c555c22c0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6713bdf2-543d-475d-a11f-286c555c22c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:14:31.034648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:31.036119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:14:31.037451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:14:31.037451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:31.037451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:31.037451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 606/1215] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:31.045401 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7c4f5-79d1-4621-a064-75d3c07dbea5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:31.128173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:31.175660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7c4f5-79d1-4621-a064-75d3c07dbea5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:31.213240 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906eaae1-1e06-447c-81a7-5ee7307b2f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:31.213887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906eaae1-1e06-447c-81a7-5ee7307b2f84 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:31.214184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906eaae1-1e06-447c-81a7-5ee7307b2f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:31.214456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906eaae1-1e06-447c-81a7-5ee7307b2f84 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:31.215096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906eaae1-1e06-447c-81a7-5ee7307b2f84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:31.215394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906eaae1-1e06-447c-81a7-5ee7307b2f84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:31.216791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906eaae1-1e06-447c-81a7-5ee7307b2f84 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:31.231019 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 610/1216] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:31.240505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713bdf2-543d-475d-a11f-286c555c22c0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:31.325696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:31.362347 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713bdf2-543d-475d-a11f-286c555c22c0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:31.400859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705db56-0970-4ea9-b19f-25d7fda1fe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:31.401485 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705db56-0970-4ea9-b19f-25d7fda1fe29 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:31.401763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705db56-0970-4ea9-b19f-25d7fda1fe29 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:31.402045 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705db56-0970-4ea9-b19f-25d7fda1fe29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:31.409128 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705db56-0970-4ea9-b19f-25d7fda1fe29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:31.410600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705db56-0970-4ea9-b19f-25d7fda1fe29 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-b705db56-0970-4ea9-b19f-25d7fda1fe29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:14:31.424390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:31.425597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:31.426950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:14:31.426950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:14:31.426950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:14:31.426950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:14:31.426950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:14:31.426950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:14:31.426950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:14:31.426950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:31.426950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 607/1217] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:14:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:31.432065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906eaae1-1e06-447c-81a7-5ee7307b2f84 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:31.541678 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906eaae1-1e06-447c-81a7-5ee7307b2f84 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:31.566246 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:31.580416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219817b3-29c7-4f7b-ad8b-c2e6f1e3dc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:31.580845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219817b3-29c7-4f7b-ad8b-c2e6f1e3dc95 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:31.581213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219817b3-29c7-4f7b-ad8b-c2e6f1e3dc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:31.581489 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219817b3-29c7-4f7b-ad8b-c2e6f1e3dc95 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:31.582208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219817b3-29c7-4f7b-ad8b-c2e6f1e3dc95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:31.582512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219817b3-29c7-4f7b-ad8b-c2e6f1e3dc95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:31.583936 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219817b3-29c7-4f7b-ad8b-c2e6f1e3dc95 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:31.598233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 611/1218] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:14:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:31.605141 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705db56-0970-4ea9-b19f-25d7fda1fe29 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:31.712998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705db56-0970-4ea9-b19f-25d7fda1fe29 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:31.744952 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:31.752260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff48035-c5b1-4ae8-b651-9a08b2f0a4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:31.752892 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff48035-c5b1-4ae8-b651-9a08b2f0a4cd None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:31.753169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff48035-c5b1-4ae8-b651-9a08b2f0a4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:31.753441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff48035-c5b1-4ae8-b651-9a08b2f0a4cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:31.754270 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff48035-c5b1-4ae8-b651-9a08b2f0a4cd None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=3b16c430b870424c8d366c9a2959003e) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:31.754580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff48035-c5b1-4ae8-b651-9a08b2f0a4cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:31.756204 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff48035-c5b1-4ae8-b651-9a08b2f0a4cd None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:31.762053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 608/1219] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:14:31 2022] GET /identity/v3/role_assignments?user.id=3b16c430b870424c8d366c9a2959003e&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:33.998391 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819fd4b-6aa8-4ce9-8257-fe74b170cb46 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:33.998948 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819fd4b-6aa8-4ce9-8257-fe74b170cb46 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:33.999069 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819fd4b-6aa8-4ce9-8257-fe74b170cb46 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:34.000519 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 612/1220] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:14:34.005745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1dca2-6d36-4d05-a065-160fa9e742bd None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:34.006218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1dca2-6d36-4d05-a065-160fa9e742bd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:34.006333 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1dca2-6d36-4d05-a065-160fa9e742bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:34.029158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7bb1dca2-6d36-4d05-a065-160fa9e742bd None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:14:34.030395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7bb1dca2-6d36-4d05-a065-160fa9e742bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:14:34.100993 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 609/1221] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:14:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:14:34.371643 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819fd4b-6aa8-4ce9-8257-fe74b170cb46 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:34.485193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819fd4b-6aa8-4ce9-8257-fe74b170cb46 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:34.527265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436ab05-ac87-43a9-9815-f6a46e76b076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:34.527883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ab05-ac87-43a9-9815-f6a46e76b076 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:34.528203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ab05-ac87-43a9-9815-f6a46e76b076 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:34.528547 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ab05-ac87-43a9-9815-f6a46e76b076 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:34.538787 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436ab05-ac87-43a9-9815-f6a46e76b076 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-3436ab05-ac87-43a9-9815-f6a46e76b076 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:14:34.544559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:34.545728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:34.546815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:34.546815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:34.546815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:34.546815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:34.546815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:14:34.546815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:14:34.546815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:14:34.546815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:14:34.546815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:14:34.546815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:14:34.546815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:14:34.546815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:34.546815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 613/1222] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:14:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:14:34.551930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1dca2-6d36-4d05-a065-160fa9e742bd None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:34.664985 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1dca2-6d36-4d05-a065-160fa9e742bd None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:34.691821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:34.704760 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331269c-eb1c-4bb7-95fd-4ca73927974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:34.705360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331269c-eb1c-4bb7-95fd-4ca73927974a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:34.705632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331269c-eb1c-4bb7-95fd-4ca73927974a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:34.705898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331269c-eb1c-4bb7-95fd-4ca73927974a None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:34.706584 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331269c-eb1c-4bb7-95fd-4ca73927974a None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:34.706903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331269c-eb1c-4bb7-95fd-4ca73927974a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:34.708225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f331269c-eb1c-4bb7-95fd-4ca73927974a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:34.716417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 610/1223] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:14:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:34.722255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436ab05-ac87-43a9-9815-f6a46e76b076 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:34.772121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:34.835756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436ab05-ac87-43a9-9815-f6a46e76b076 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:34.877280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afaa29-352d-44dd-b967-522ad17b9a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:34.878008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afaa29-352d-44dd-b967-522ad17b9a83 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:34.878320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afaa29-352d-44dd-b967-522ad17b9a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:34.878590 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afaa29-352d-44dd-b967-522ad17b9a83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:34.891857 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70afaa29-352d-44dd-b967-522ad17b9a83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:34.893481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70afaa29-352d-44dd-b967-522ad17b9a83 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-70afaa29-352d-44dd-b967-522ad17b9a83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:14:34.910364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:34.911482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:14:34.912695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:14:34.912695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:34.912695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:34.912695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 614/1224] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:34.917418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331269c-eb1c-4bb7-95fd-4ca73927974a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:35.028278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331269c-eb1c-4bb7-95fd-4ca73927974a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:35.066011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6884964-1f23-477f-9559-6b83376e3512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:35.066697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6884964-1f23-477f-9559-6b83376e3512 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:35.067012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6884964-1f23-477f-9559-6b83376e3512 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:35.067318 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6884964-1f23-477f-9559-6b83376e3512 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:35.068042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6884964-1f23-477f-9559-6b83376e3512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:35.068332 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6884964-1f23-477f-9559-6b83376e3512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:35.069833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6884964-1f23-477f-9559-6b83376e3512 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:35.082827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 611/1225] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:35.088730 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afaa29-352d-44dd-b967-522ad17b9a83 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:35.202592 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:35.209947 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afaa29-352d-44dd-b967-522ad17b9a83 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:35.255193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91012db-48d1-4d5a-831c-113b64dba995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:35.256281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91012db-48d1-4d5a-831c-113b64dba995 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:35.256779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91012db-48d1-4d5a-831c-113b64dba995 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:35.257119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91012db-48d1-4d5a-831c-113b64dba995 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:35.266637 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91012db-48d1-4d5a-831c-113b64dba995 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:35.268101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91012db-48d1-4d5a-831c-113b64dba995 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d91012db-48d1-4d5a-831c-113b64dba995 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:14:35.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:35.282276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:35.283234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:35.283234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:35.283234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:14:35.283234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:14:35.283234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:14:35.283234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:14:35.283234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:14:35.283234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:14:35.283234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 27 17:14:35.283234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:35.283234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 615/1226] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:14:35.288393 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6884964-1f23-477f-9559-6b83376e3512 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:35.340764 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:35.399205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6884964-1f23-477f-9559-6b83376e3512 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:35.437426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12ae03-114f-4aa1-997e-df410276d0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:35.438035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12ae03-114f-4aa1-997e-df410276d0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:35.438348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12ae03-114f-4aa1-997e-df410276d0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:35.438669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12ae03-114f-4aa1-997e-df410276d0e8 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:35.439380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12ae03-114f-4aa1-997e-df410276d0e8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:35.439657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12ae03-114f-4aa1-997e-df410276d0e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:35.441081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12ae03-114f-4aa1-997e-df410276d0e8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:35.459860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 612/1227] 10.222.0.55 () {56 vars in 1143 bytes} [Tue Sep 27 17:14:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:14:35.465857 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91012db-48d1-4d5a-831c-113b64dba995 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:35.569445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91012db-48d1-4d5a-831c-113b64dba995 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:35.609886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f58927-b2f5-455d-a954-0cb518fbdfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:35.610709 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f58927-b2f5-455d-a954-0cb518fbdfe8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:35.611072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f58927-b2f5-455d-a954-0cb518fbdfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:35.611349 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f58927-b2f5-455d-a954-0cb518fbdfe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:35.626540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f58927-b2f5-455d-a954-0cb518fbdfe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:35.627994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f58927-b2f5-455d-a954-0cb518fbdfe8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-17f58927-b2f5-455d-a954-0cb518fbdfe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:14:35.644424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:35.645554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:14:35.651728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:14:35.651728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:35.651728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:35.651728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 616/1228] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:35.651986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12ae03-114f-4aa1-997e-df410276d0e8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:35.766890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12ae03-114f-4aa1-997e-df410276d0e8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:35.769316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:35.805283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6a588-edd6-4f47-8375-9213fc8f3d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:35.806041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6a588-edd6-4f47-8375-9213fc8f3d9e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:35.806440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6a588-edd6-4f47-8375-9213fc8f3d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:35.806915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6a588-edd6-4f47-8375-9213fc8f3d9e None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:35.807731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6a588-edd6-4f47-8375-9213fc8f3d9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:35.808191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6a588-edd6-4f47-8375-9213fc8f3d9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:35.810339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6a588-edd6-4f47-8375-9213fc8f3d9e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:35.824382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 613/1229] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:35.832357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f58927-b2f5-455d-a954-0cb518fbdfe8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:35.847786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:35.944958 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f58927-b2f5-455d-a954-0cb518fbdfe8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:35.987013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d6de1-d10d-4a6e-bf71-9bce45c699ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:35.987835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d6de1-d10d-4a6e-bf71-9bce45c699ce None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:35.988203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d6de1-d10d-4a6e-bf71-9bce45c699ce None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:35.988510 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d6de1-d10d-4a6e-bf71-9bce45c699ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:35.998591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d6de1-d10d-4a6e-bf71-9bce45c699ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:36.000894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765d6de1-d10d-4a6e-bf71-9bce45c699ce None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-765d6de1-d10d-4a6e-bf71-9bce45c699ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:14:36.011900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:36.014294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:36.015332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:14:36.015332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:14:36.015332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:14:36.015332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:14:36.015332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:14:36.015332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:14:36.015332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:14:36.015332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:36.015332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 617/1230] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:14:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:14:36.018988 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6a588-edd6-4f47-8375-9213fc8f3d9e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:36.130830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6a588-edd6-4f47-8375-9213fc8f3d9e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:36.167014 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b7324-efaa-45c8-bffd-a1f4fe3a4080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:36.167618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b7324-efaa-45c8-bffd-a1f4fe3a4080 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:36.167893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b7324-efaa-45c8-bffd-a1f4fe3a4080 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:36.168160 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b7324-efaa-45c8-bffd-a1f4fe3a4080 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:36.168849 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b7324-efaa-45c8-bffd-a1f4fe3a4080 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:36.169154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b7324-efaa-45c8-bffd-a1f4fe3a4080 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:36.170599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b7324-efaa-45c8-bffd-a1f4fe3a4080 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:36.188608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 614/1231] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:14:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:36.189759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d6de1-d10d-4a6e-bf71-9bce45c699ce None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:36.261552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:36.303026 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d6de1-d10d-4a6e-bf71-9bce45c699ce None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:36.338393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f91fc46-d46b-4ed5-881e-53371de66b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:36.339300 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91fc46-d46b-4ed5-881e-53371de66b26 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:36.339581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91fc46-d46b-4ed5-881e-53371de66b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:36.339856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91fc46-d46b-4ed5-881e-53371de66b26 None admin] PATH_INFO: `/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/3b16c430b870424c8d366c9a2959003e/roles/fafa3faaeb1344d2b6f639709d49e59b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:36.355631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f91fc46-d46b-4ed5-881e-53371de66b26 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1f2574a4344711a56722bca143ae6c, user_id=3b16c430b870424c8d366c9a2959003e, role_id=fafa3faaeb1344d2b6f639709d49e59b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:36.357239 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f91fc46-d46b-4ed5-881e-53371de66b26 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:36.370513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 618/1232] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:14:36 2022] PUT /identity/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/3b16c430b870424c8d366c9a2959003e/roles/fafa3faaeb1344d2b6f639709d49e59b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:14:38.707604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc3504-3500-4cb9-b44b-076cba2bf353 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:38.708266 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc3504-3500-4cb9-b44b-076cba2bf353 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:38.708266 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc3504-3500-4cb9-b44b-076cba2bf353 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:38.709708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 615/1233] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:14:38.716406 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ee52b-4e64-4c08-b4d8-7f62923f784e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:38.716914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ee52b-4e64-4c08-b4d8-7f62923f784e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:38.717019 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ee52b-4e64-4c08-b4d8-7f62923f784e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:38.736248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ad8ee52b-4e64-4c08-b4d8-7f62923f784e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:14:38.737503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ad8ee52b-4e64-4c08-b4d8-7f62923f784e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:14:38.810613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 619/1234] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:14:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:14:39.106165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc3504-3500-4cb9-b44b-076cba2bf353 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:39.136678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:39.223144 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc3504-3500-4cb9-b44b-076cba2bf353 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:39.262268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471ad8b-36ec-460e-9817-53843b6b6db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:39.262914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471ad8b-36ec-460e-9817-53843b6b6db4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:39.263318 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471ad8b-36ec-460e-9817-53843b6b6db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:39.263544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471ad8b-36ec-460e-9817-53843b6b6db4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:39.276251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471ad8b-36ec-460e-9817-53843b6b6db4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-d471ad8b-36ec-460e-9817-53843b6b6db4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:14:39.282130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:39.283323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:39.284471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:39.284471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:39.284471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:39.284471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:39.284471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:14:39.284471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:14:39.284471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:14:39.284471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:14:39.284471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:14:39.284471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:14:39.284471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 27 17:14:39.284471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:39.284471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 616/1235] 10.222.0.55 () {56 vars in 1102 bytes} [Tue Sep 27 17:14:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 27 17:14:39.289634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ee52b-4e64-4c08-b4d8-7f62923f784e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:39.339168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:39.394205 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ee52b-4e64-4c08-b4d8-7f62923f784e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:39.432803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871c52e-dde1-418c-aafc-1392695898d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:39.433561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871c52e-dde1-418c-aafc-1392695898d4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:39.433786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871c52e-dde1-418c-aafc-1392695898d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:39.434143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871c52e-dde1-418c-aafc-1392695898d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:39.434872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871c52e-dde1-418c-aafc-1392695898d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:39.435229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871c52e-dde1-418c-aafc-1392695898d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:39.436835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871c52e-dde1-418c-aafc-1392695898d4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:39.450848 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 620/1236] 10.222.0.55 () {56 vars in 1103 bytes} [Tue Sep 27 17:14:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:39.452899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471ad8b-36ec-460e-9817-53843b6b6db4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:39.547490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:39.566345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471ad8b-36ec-460e-9817-53843b6b6db4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:39.605180 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf6462-27c7-4e5f-8a48-c7448897ea11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:39.605918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf6462-27c7-4e5f-8a48-c7448897ea11 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:39.606286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf6462-27c7-4e5f-8a48-c7448897ea11 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:39.606567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf6462-27c7-4e5f-8a48-c7448897ea11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:39.619033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf6462-27c7-4e5f-8a48-c7448897ea11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:39.620724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf6462-27c7-4e5f-8a48-c7448897ea11 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-aecf6462-27c7-4e5f-8a48-c7448897ea11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:14:39.636000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:39.637268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:14:39.638492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:14:39.638492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:39.638492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:39.638492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 617/1237] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:39.643282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871c52e-dde1-418c-aafc-1392695898d4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:39.756990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871c52e-dde1-418c-aafc-1392695898d4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:39.767347 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:39.794999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef6e5e-5d98-44ad-a232-b64f645b1f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:39.795685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6e5e-5d98-44ad-a232-b64f645b1f93 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:39.795965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6e5e-5d98-44ad-a232-b64f645b1f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:39.796235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef6e5e-5d98-44ad-a232-b64f645b1f93 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:39.796875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef6e5e-5d98-44ad-a232-b64f645b1f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:39.797173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef6e5e-5d98-44ad-a232-b64f645b1f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:39.798595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef6e5e-5d98-44ad-a232-b64f645b1f93 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:39.811472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 621/1238] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:39.817318 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf6462-27c7-4e5f-8a48-c7448897ea11 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:39.928603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf6462-27c7-4e5f-8a48-c7448897ea11 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:39.968488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65de0a-f781-429f-9e2f-dc6dec1e6fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:39.969582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65de0a-f781-429f-9e2f-dc6dec1e6fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:39.969994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65de0a-f781-429f-9e2f-dc6dec1e6fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:39.970348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65de0a-f781-429f-9e2f-dc6dec1e6fc7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:39.974125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:39.983423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb65de0a-f781-429f-9e2f-dc6dec1e6fc7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:39.984912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb65de0a-f781-429f-9e2f-dc6dec1e6fc7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-bb65de0a-f781-429f-9e2f-dc6dec1e6fc7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:14:40.000056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:40.001297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:40.002517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:40.002517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:40.002517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:14:40.002517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:14:40.002517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:14:40.002517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:14:40.002517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:14:40.002517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:14:40.002517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 27 17:14:40.002517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:40.002517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 618/1239] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:40.007477 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef6e5e-5d98-44ad-a232-b64f645b1f93 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:40.120033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef6e5e-5d98-44ad-a232-b64f645b1f93 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:40.159023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19961fb-1fcb-4a76-a65e-419da56b6f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:40.159626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19961fb-1fcb-4a76-a65e-419da56b6f24 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:40.159899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19961fb-1fcb-4a76-a65e-419da56b6f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:40.160174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19961fb-1fcb-4a76-a65e-419da56b6f24 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:40.160907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19961fb-1fcb-4a76-a65e-419da56b6f24 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:40.161197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19961fb-1fcb-4a76-a65e-419da56b6f24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:40.162760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19961fb-1fcb-4a76-a65e-419da56b6f24 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:40.182065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 622/1240] 10.222.0.55 () {56 vars in 1143 bytes} [Tue Sep 27 17:14:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:40.187958 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65de0a-f781-429f-9e2f-dc6dec1e6fc7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:40.296947 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65de0a-f781-429f-9e2f-dc6dec1e6fc7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:40.333984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc17ab8-144b-45da-81d1-100d701d2493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:40.334735 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc17ab8-144b-45da-81d1-100d701d2493 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:40.335115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc17ab8-144b-45da-81d1-100d701d2493 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:40.335535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc17ab8-144b-45da-81d1-100d701d2493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:40.403907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:40.411691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc17ab8-144b-45da-81d1-100d701d2493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:40.413144 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc17ab8-144b-45da-81d1-100d701d2493 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:40.526133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 619/1241] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:14:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:14:40.530986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19961fb-1fcb-4a76-a65e-419da56b6f24 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:40.562837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:40.644197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19961fb-1fcb-4a76-a65e-419da56b6f24 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:40.687594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5a95f-435a-4c0b-94ea-3df25f3dd2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:40.688361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5a95f-435a-4c0b-94ea-3df25f3dd2fc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:40.688814 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5a95f-435a-4c0b-94ea-3df25f3dd2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:40.689275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5a95f-435a-4c0b-94ea-3df25f3dd2fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:40.705092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5a95f-435a-4c0b-94ea-3df25f3dd2fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:40.707340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5a95f-435a-4c0b-94ea-3df25f3dd2fc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c7c5a95f-435a-4c0b-94ea-3df25f3dd2fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:14:40.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:40.729974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:14:40.730932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:14:40.730932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:40.730932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:40.730932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 623/1242] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:40.737163 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc17ab8-144b-45da-81d1-100d701d2493 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:40.835001 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:40.842059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc17ab8-144b-45da-81d1-100d701d2493 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:40.880055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb9b64-9ed8-437b-88dc-51f392d97808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:40.880819 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb9b64-9ed8-437b-88dc-51f392d97808 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:40.881217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb9b64-9ed8-437b-88dc-51f392d97808 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:40.881578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb9b64-9ed8-437b-88dc-51f392d97808 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:40.882487 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb9b64-9ed8-437b-88dc-51f392d97808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:40.882829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb9b64-9ed8-437b-88dc-51f392d97808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:40.884448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb9b64-9ed8-437b-88dc-51f392d97808 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:40.897058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 620/1243] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:14:40.903784 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5a95f-435a-4c0b-94ea-3df25f3dd2fc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:40.999248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:41.026423 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5a95f-435a-4c0b-94ea-3df25f3dd2fc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:41.065361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7d992-caa4-4422-a2e3-31920919b100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:41.065990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7d992-caa4-4422-a2e3-31920919b100 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:41.066301 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7d992-caa4-4422-a2e3-31920919b100 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:41.066580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7d992-caa4-4422-a2e3-31920919b100 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:41.074591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7d992-caa4-4422-a2e3-31920919b100 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:41.076011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e7d992-caa4-4422-a2e3-31920919b100 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-97e7d992-caa4-4422-a2e3-31920919b100 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:14:41.088355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:41.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:41.090532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:14:41.090532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:14:41.090532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:14:41.090532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:14:41.090532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:14:41.090532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:14:41.090532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:14:41.090532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:41.090532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 624/1244] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:14:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:41.095934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb9b64-9ed8-437b-88dc-51f392d97808 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:41.229428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb9b64-9ed8-437b-88dc-51f392d97808 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:41.269989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17b4fb-e6e1-42d4-8d4d-c38dff6190b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:41.270685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17b4fb-e6e1-42d4-8d4d-c38dff6190b1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:41.271059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17b4fb-e6e1-42d4-8d4d-c38dff6190b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:41.271426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17b4fb-e6e1-42d4-8d4d-c38dff6190b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:41.272220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17b4fb-e6e1-42d4-8d4d-c38dff6190b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:41.272620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17b4fb-e6e1-42d4-8d4d-c38dff6190b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:41.274838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17b4fb-e6e1-42d4-8d4d-c38dff6190b1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:41.290159 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 621/1245] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:14:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:41.296486 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7d992-caa4-4422-a2e3-31920919b100 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:41.424022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e7d992-caa4-4422-a2e3-31920919b100 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:41.467043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1a72c-caed-4f8a-8565-1628894984f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:41.467662 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1a72c-caed-4f8a-8565-1628894984f0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:41.467943 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1a72c-caed-4f8a-8565-1628894984f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:41.468244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1a72c-caed-4f8a-8565-1628894984f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:41.469162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1a72c-caed-4f8a-8565-1628894984f0 None admin] RBAC: Adding query filter params (role.id=fafa3faaeb1344d2b6f639709d49e59b, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=3b16c430b870424c8d366c9a2959003e) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:41.469505 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1a72c-caed-4f8a-8565-1628894984f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:41.471233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1a72c-caed-4f8a-8565-1628894984f0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:41.477933 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 625/1246] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:14:41 2022] GET /identity/v3/role_assignments?user.id=3b16c430b870424c8d366c9a2959003e&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=fafa3faaeb1344d2b6f639709d49e59b => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:14:43.881990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd44d4-b9ea-4e58-afef-e7548a72727c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:43.883107 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd44d4-b9ea-4e58-afef-e7548a72727c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:43.883107 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd44d4-b9ea-4e58-afef-e7548a72727c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:43.884466 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 622/1247] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:14:43.890621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6b0e2-22d6-4290-a11d-37710b88bfe7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:43.891047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6b0e2-22d6-4290-a11d-37710b88bfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:43.891256 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6b0e2-22d6-4290-a11d-37710b88bfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:43.898649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:43.911726 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f8a6b0e2-22d6-4290-a11d-37710b88bfe7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:14:43.912891 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f8a6b0e2-22d6-4290-a11d-37710b88bfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:14:43.988827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 626/1248] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:14:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:14:44.294518 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd44d4-b9ea-4e58-afef-e7548a72727c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:44.321228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:44.432018 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd44d4-b9ea-4e58-afef-e7548a72727c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:44.468527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b545f3-260f-45d4-8d40-8339b327ad20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:44.469174 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b545f3-260f-45d4-8d40-8339b327ad20 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:44.469451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b545f3-260f-45d4-8d40-8339b327ad20 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:44.469720 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b545f3-260f-45d4-8d40-8339b327ad20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:44.482004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b545f3-260f-45d4-8d40-8339b327ad20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-67b545f3-260f-45d4-8d40-8339b327ad20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:14:44.486855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:44.487872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:44.488921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:44.488921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:44.488921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:44.488921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:44.488921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:14:44.488921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:14:44.488921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:14:44.488921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:14:44.488921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:14:44.488921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:14:44.488921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:14:44.488921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:44.488921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 623/1249] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:44.494897 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6b0e2-22d6-4290-a11d-37710b88bfe7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:44.597002 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6b0e2-22d6-4290-a11d-37710b88bfe7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:44.628122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:44.634194 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78069bcc-2422-4a51-a228-7d5861de073a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:44.634816 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78069bcc-2422-4a51-a228-7d5861de073a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:44.635092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78069bcc-2422-4a51-a228-7d5861de073a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:44.635361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78069bcc-2422-4a51-a228-7d5861de073a None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:44.636034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78069bcc-2422-4a51-a228-7d5861de073a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:44.636336 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78069bcc-2422-4a51-a228-7d5861de073a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:44.637711 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78069bcc-2422-4a51-a228-7d5861de073a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:44.646987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 627/1250] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:44.652984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b545f3-260f-45d4-8d40-8339b327ad20 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:44.734382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:44.758696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b545f3-260f-45d4-8d40-8339b327ad20 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:44.796921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f186a32-c7fc-4a9e-906a-5adf038ec0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:44.797680 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f186a32-c7fc-4a9e-906a-5adf038ec0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:44.798128 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f186a32-c7fc-4a9e-906a-5adf038ec0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:44.799534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f186a32-c7fc-4a9e-906a-5adf038ec0f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:44.810309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f186a32-c7fc-4a9e-906a-5adf038ec0f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:44.811757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f186a32-c7fc-4a9e-906a-5adf038ec0f7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-5f186a32-c7fc-4a9e-906a-5adf038ec0f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:14:44.826156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:44.827360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:14:44.828514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:14:44.828514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:44.828514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:44.828514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 624/1251] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:44.833197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78069bcc-2422-4a51-a228-7d5861de073a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:44.942830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78069bcc-2422-4a51-a228-7d5861de073a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:44.982014 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adf286-c7b1-4493-91b9-1385d383fb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:44.982751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adf286-c7b1-4493-91b9-1385d383fb49 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:44.982989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adf286-c7b1-4493-91b9-1385d383fb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:44.983333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adf286-c7b1-4493-91b9-1385d383fb49 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:44.984073 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81adf286-c7b1-4493-91b9-1385d383fb49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:44.984456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81adf286-c7b1-4493-91b9-1385d383fb49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:44.986205 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81adf286-c7b1-4493-91b9-1385d383fb49 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:45.001086 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 628/1252] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:45.007259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f186a32-c7fc-4a9e-906a-5adf038ec0f7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:45.113467 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f186a32-c7fc-4a9e-906a-5adf038ec0f7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:45.139892 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:45.149035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98d3d9-1b30-4854-996e-76ee14dc9006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:45.149707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98d3d9-1b30-4854-996e-76ee14dc9006 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:45.149996 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98d3d9-1b30-4854-996e-76ee14dc9006 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:45.150314 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98d3d9-1b30-4854-996e-76ee14dc9006 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:45.157927 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98d3d9-1b30-4854-996e-76ee14dc9006 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:45.159508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98d3d9-1b30-4854-996e-76ee14dc9006 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-9b98d3d9-1b30-4854-996e-76ee14dc9006 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:14:45.170974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:45.172023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:45.172989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:45.172989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:45.172989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:14:45.172989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:14:45.172989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:14:45.172989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:14:45.172989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:14:45.172989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:14:45.172989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 27 17:14:45.172989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:45.172989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 625/1253] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:45.178087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adf286-c7b1-4493-91b9-1385d383fb49 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:45.231817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:45.274922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81adf286-c7b1-4493-91b9-1385d383fb49 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:45.306594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b7dd9-0acd-4c1a-8e99-98c84b8b7487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:45.307176 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b7dd9-0acd-4c1a-8e99-98c84b8b7487 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:45.307445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b7dd9-0acd-4c1a-8e99-98c84b8b7487 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:45.307708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b7dd9-0acd-4c1a-8e99-98c84b8b7487 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:45.308417 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b7dd9-0acd-4c1a-8e99-98c84b8b7487 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:45.308709 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b7dd9-0acd-4c1a-8e99-98c84b8b7487 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:45.310129 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b7dd9-0acd-4c1a-8e99-98c84b8b7487 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:45.327796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 629/1254] 10.222.0.55 () {56 vars in 1143 bytes} [Tue Sep 27 17:14:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:45.333652 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98d3d9-1b30-4854-996e-76ee14dc9006 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:45.429470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98d3d9-1b30-4854-996e-76ee14dc9006 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:45.461946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28004ca9-d825-41ef-9629-2c25a21d7f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:45.462568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28004ca9-d825-41ef-9629-2c25a21d7f07 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:45.462845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28004ca9-d825-41ef-9629-2c25a21d7f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:45.463115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28004ca9-d825-41ef-9629-2c25a21d7f07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:45.516547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:45.529548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28004ca9-d825-41ef-9629-2c25a21d7f07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:45.531056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28004ca9-d825-41ef-9629-2c25a21d7f07 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:45.624741 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 626/1255] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:14:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:14:45.630565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b7dd9-0acd-4c1a-8e99-98c84b8b7487 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:45.725985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b7dd9-0acd-4c1a-8e99-98c84b8b7487 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:45.738804 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:45.760395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfab14f-48f1-460d-91d7-94815db02575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:45.760999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfab14f-48f1-460d-91d7-94815db02575 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:45.761275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfab14f-48f1-460d-91d7-94815db02575 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:45.761547 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfab14f-48f1-460d-91d7-94815db02575 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:45.771350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfab14f-48f1-460d-91d7-94815db02575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:45.772709 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfab14f-48f1-460d-91d7-94815db02575 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1dfab14f-48f1-460d-91d7-94815db02575 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:14:45.791639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:45.792795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:14:45.793697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:14:45.793697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:45.793697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:45.793697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 630/1256] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:45.798436 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28004ca9-d825-41ef-9629-2c25a21d7f07 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:45.886268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:45.898369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28004ca9-d825-41ef-9629-2c25a21d7f07 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:45.931550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499da23-a2bd-4df0-a3c4-33b0ef021a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:45.932138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499da23-a2bd-4df0-a3c4-33b0ef021a8d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:45.932414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499da23-a2bd-4df0-a3c4-33b0ef021a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:45.932683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499da23-a2bd-4df0-a3c4-33b0ef021a8d None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:45.933316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499da23-a2bd-4df0-a3c4-33b0ef021a8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:45.933613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499da23-a2bd-4df0-a3c4-33b0ef021a8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:45.934989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499da23-a2bd-4df0-a3c4-33b0ef021a8d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:45.945738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 627/1257] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:45.951341 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfab14f-48f1-460d-91d7-94815db02575 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:46.054233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfab14f-48f1-460d-91d7-94815db02575 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:46.086224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6542f0-e120-4c3f-90dc-8464f463651c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:46.086837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6542f0-e120-4c3f-90dc-8464f463651c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:46.087116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6542f0-e120-4c3f-90dc-8464f463651c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:46.087385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6542f0-e120-4c3f-90dc-8464f463651c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:46.093757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6542f0-e120-4c3f-90dc-8464f463651c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:46.095214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6542f0-e120-4c3f-90dc-8464f463651c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-4d6542f0-e120-4c3f-90dc-8464f463651c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:14:46.106878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:46.107953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:46.109124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:14:46.109124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:14:46.109124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:14:46.109124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:14:46.109124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:14:46.109124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:14:46.109124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:14:46.109124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:46.109124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 631/1258] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:14:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:46.113965 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499da23-a2bd-4df0-a3c4-33b0ef021a8d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:46.209259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499da23-a2bd-4df0-a3c4-33b0ef021a8d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:46.242427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd427e-7725-4bde-951f-75f3427a80e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:46.243018 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd427e-7725-4bde-951f-75f3427a80e5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:46.243293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd427e-7725-4bde-951f-75f3427a80e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:46.243561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd427e-7725-4bde-951f-75f3427a80e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:46.244246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd427e-7725-4bde-951f-75f3427a80e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:46.244547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd427e-7725-4bde-951f-75f3427a80e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:46.245958 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd427e-7725-4bde-951f-75f3427a80e5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:46.257469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 628/1259] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:14:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:46.263442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6542f0-e120-4c3f-90dc-8464f463651c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:46.288672 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:46.361999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6542f0-e120-4c3f-90dc-8464f463651c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:46.393433 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352af432-5f72-4383-b617-186ad3f86b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:46.394019 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352af432-5f72-4383-b617-186ad3f86b77 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:46.394332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352af432-5f72-4383-b617-186ad3f86b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:46.394604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352af432-5f72-4383-b617-186ad3f86b77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:46.395317 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352af432-5f72-4383-b617-186ad3f86b77 None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=3b16c430b870424c8d366c9a2959003e) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:46.395623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352af432-5f72-4383-b617-186ad3f86b77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:46.397029 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352af432-5f72-4383-b617-186ad3f86b77 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:46.402843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 632/1260] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:14:46 2022] GET /identity/v3/role_assignments?user.id=3b16c430b870424c8d366c9a2959003e&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:48.576539 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5363752-0bf7-4e15-a8e0-b4af8bb363d9 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:48.577093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5363752-0bf7-4e15-a8e0-b4af8bb363d9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:48.577093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5363752-0bf7-4e15-a8e0-b4af8bb363d9 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:48.578360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 629/1261] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:14:48.583516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517df8ec-c630-4836-a07c-185cefa93bab None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:48.583922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517df8ec-c630-4836-a07c-185cefa93bab None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:48.584064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517df8ec-c630-4836-a07c-185cefa93bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:48.602916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-517df8ec-c630-4836-a07c-185cefa93bab None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:14:48.604343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-517df8ec-c630-4836-a07c-185cefa93bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:14:48.670128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 633/1262] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:14:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:14:48.939233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5363752-0bf7-4e15-a8e0-b4af8bb363d9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:48.950435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:49.052512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5363752-0bf7-4e15-a8e0-b4af8bb363d9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:49.088841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0e563-81ad-4dca-ace6-14723e45fd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:49.089455 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0e563-81ad-4dca-ace6-14723e45fd37 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:49.089727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0e563-81ad-4dca-ace6-14723e45fd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:49.089989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0e563-81ad-4dca-ace6-14723e45fd37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:49.099623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe0e563-81ad-4dca-ace6-14723e45fd37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-3fe0e563-81ad-4dca-ace6-14723e45fd37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:14:49.104564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:49.105716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:49.107016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:49.107016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:49.107016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:49.107016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:49.107016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:14:49.107016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:14:49.107016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:14:49.107016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:14:49.107016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:14:49.107016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:14:49.107016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:14:49.107016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:49.107016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 630/1263] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:49.111701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517df8ec-c630-4836-a07c-185cefa93bab None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:49.128993 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:49.206542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517df8ec-c630-4836-a07c-185cefa93bab None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:49.239037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c3aa6-08ed-403d-8d7e-27e0eb443d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:49.239620 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c3aa6-08ed-403d-8d7e-27e0eb443d32 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:49.239896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c3aa6-08ed-403d-8d7e-27e0eb443d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:49.240166 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c3aa6-08ed-403d-8d7e-27e0eb443d32 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:49.240816 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c3aa6-08ed-403d-8d7e-27e0eb443d32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:49.241113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c3aa6-08ed-403d-8d7e-27e0eb443d32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:49.242522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c3aa6-08ed-403d-8d7e-27e0eb443d32 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:49.250206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 634/1264] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:14:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:49.255807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0e563-81ad-4dca-ace6-14723e45fd37 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:49.322419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:49.355675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0e563-81ad-4dca-ace6-14723e45fd37 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:49.389753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c141a1f-61b4-41ea-a999-527f8882d324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:49.390393 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c141a1f-61b4-41ea-a999-527f8882d324 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:49.390672 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c141a1f-61b4-41ea-a999-527f8882d324 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:49.390941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c141a1f-61b4-41ea-a999-527f8882d324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:49.400615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c141a1f-61b4-41ea-a999-527f8882d324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:49.402047 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c141a1f-61b4-41ea-a999-527f8882d324 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-8c141a1f-61b4-41ea-a999-527f8882d324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:14:49.416113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:49.417226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:14:49.418178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:14:49.418178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:49.418178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:49.418178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 631/1265] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:49.423457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c3aa6-08ed-403d-8d7e-27e0eb443d32 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:49.501625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:49.521055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c3aa6-08ed-403d-8d7e-27e0eb443d32 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:49.552889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2982b9-f718-4b7f-b09b-432b16531ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:49.553471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2982b9-f718-4b7f-b09b-432b16531ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:49.553745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2982b9-f718-4b7f-b09b-432b16531ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:49.554558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2982b9-f718-4b7f-b09b-432b16531ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:49.555199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2982b9-f718-4b7f-b09b-432b16531ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:49.555495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2982b9-f718-4b7f-b09b-432b16531ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:49.556846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2982b9-f718-4b7f-b09b-432b16531ed6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:49.568061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 635/1266] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:49.574426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c141a1f-61b4-41ea-a999-527f8882d324 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:49.669450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c141a1f-61b4-41ea-a999-527f8882d324 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:49.688654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:49.702417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32b9bc-1a4d-4b71-9dca-03705f9e3bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:49.703031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32b9bc-1a4d-4b71-9dca-03705f9e3bff None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:49.703310 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32b9bc-1a4d-4b71-9dca-03705f9e3bff None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:49.703576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32b9bc-1a4d-4b71-9dca-03705f9e3bff None admin] PATH_INFO: `/v3/users/placement` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:49.710673 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32b9bc-1a4d-4b71-9dca-03705f9e3bff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:49.712104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32b9bc-1a4d-4b71-9dca-03705f9e3bff None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6b32b9bc-1a4d-4b71-9dca-03705f9e3bff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:14:49.722846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:49.723946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:49.724923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:49.724923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:49.724923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:14:49.724923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:14:49.724923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:14:49.724923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:14:49.724923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:14:49.724923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:14:49.724923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 27 17:14:49.724923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:49.724923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 632/1267] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:49.730464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2982b9-f718-4b7f-b09b-432b16531ed6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:49.833292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2982b9-f718-4b7f-b09b-432b16531ed6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:49.866008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea41f6-f9e3-4f48-8e5a-ee0e41a23951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:49.866648 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea41f6-f9e3-4f48-8e5a-ee0e41a23951 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:49.866925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea41f6-f9e3-4f48-8e5a-ee0e41a23951 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:49.867192 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea41f6-f9e3-4f48-8e5a-ee0e41a23951 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:49.867907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea41f6-f9e3-4f48-8e5a-ee0e41a23951 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:49.868210 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea41f6-f9e3-4f48-8e5a-ee0e41a23951 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:49.869605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea41f6-f9e3-4f48-8e5a-ee0e41a23951 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:49.877855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:49.887212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 636/1268] 10.222.0.55 () {56 vars in 1143 bytes} [Tue Sep 27 17:14:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:49.892879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32b9bc-1a4d-4b71-9dca-03705f9e3bff None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:49.995519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32b9bc-1a4d-4b71-9dca-03705f9e3bff None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:50.029672 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71525c-863e-4144-b58b-26aac39a0e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:50.030309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71525c-863e-4144-b58b-26aac39a0e84 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:50.030585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71525c-863e-4144-b58b-26aac39a0e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:50.030852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71525c-863e-4144-b58b-26aac39a0e84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:50.044222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71525c-863e-4144-b58b-26aac39a0e84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:50.045663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71525c-863e-4144-b58b-26aac39a0e84 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ff71525c-863e-4144-b58b-26aac39a0e84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:14:50.059603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:50.060781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:14:50.061840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:14:50.061840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:50.061840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:50.061840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 633/1269] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:50.066459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea41f6-f9e3-4f48-8e5a-ee0e41a23951 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:50.162025 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea41f6-f9e3-4f48-8e5a-ee0e41a23951 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:50.194506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e716efd-55b0-4f2a-b04b-60ac28f04f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:50.195665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716efd-55b0-4f2a-b04b-60ac28f04f6d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:50.195940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716efd-55b0-4f2a-b04b-60ac28f04f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:50.196209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e716efd-55b0-4f2a-b04b-60ac28f04f6d None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:50.196839 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e716efd-55b0-4f2a-b04b-60ac28f04f6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:50.197138 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e716efd-55b0-4f2a-b04b-60ac28f04f6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:50.198524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e716efd-55b0-4f2a-b04b-60ac28f04f6d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:50.209149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 637/1270] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:50.214893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71525c-863e-4144-b58b-26aac39a0e84 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:50.246727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:50.313238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71525c-863e-4144-b58b-26aac39a0e84 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:50.346327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f0958-4d28-4316-b597-80a8ae2f3cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:50.346940 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f0958-4d28-4316-b597-80a8ae2f3cad None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:50.347219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f0958-4d28-4316-b597-80a8ae2f3cad None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:50.347489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f0958-4d28-4316-b597-80a8ae2f3cad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:50.354357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f0958-4d28-4316-b597-80a8ae2f3cad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:50.355830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f0958-4d28-4316-b597-80a8ae2f3cad None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-fa7f0958-4d28-4316-b597-80a8ae2f3cad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:14:50.368021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:50.369095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:50.370421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:14:50.370421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:14:50.370421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:14:50.370421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:14:50.370421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:14:50.370421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:14:50.370421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:14:50.370421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:50.370421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 634/1271] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:14:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:50.375187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e716efd-55b0-4f2a-b04b-60ac28f04f6d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:50.432575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:50.473548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e716efd-55b0-4f2a-b04b-60ac28f04f6d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:50.506522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0107840-be23-45f7-bcac-be44974d35a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:50.507134 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0107840-be23-45f7-bcac-be44974d35a2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:50.507418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0107840-be23-45f7-bcac-be44974d35a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:50.507684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0107840-be23-45f7-bcac-be44974d35a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:50.508368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0107840-be23-45f7-bcac-be44974d35a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:50.508665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0107840-be23-45f7-bcac-be44974d35a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:50.510104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0107840-be23-45f7-bcac-be44974d35a2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:50.521578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 638/1272] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:14:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:50.527688 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f0958-4d28-4316-b597-80a8ae2f3cad None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:50.617443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:50.626307 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f0958-4d28-4316-b597-80a8ae2f3cad None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:50.657910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51102550-27dc-413f-88ab-2226b2bea145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:50.658582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51102550-27dc-413f-88ab-2226b2bea145 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:50.658859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51102550-27dc-413f-88ab-2226b2bea145 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:50.659130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51102550-27dc-413f-88ab-2226b2bea145 None admin] PATH_INFO: `/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/3b16c430b870424c8d366c9a2959003e/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:50.661015 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51102550-27dc-413f-88ab-2226b2bea145 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb1f2574a4344711a56722bca143ae6c, user_id=3b16c430b870424c8d366c9a2959003e, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:50.662465 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51102550-27dc-413f-88ab-2226b2bea145 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:50.675034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 635/1273] 10.222.0.55 () {58 vars in 1439 bytes} [Tue Sep 27 17:14:50 2022] PUT /identity/v3/projects/bb1f2574a4344711a56722bca143ae6c/users/3b16c430b870424c8d366c9a2959003e/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:14:52.805187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb7191-c20e-43f8-973f-a836c59ed802 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:52.805712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb7191-c20e-43f8-973f-a836c59ed802 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:52.805712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb7191-c20e-43f8-973f-a836c59ed802 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:52.807100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 639/1274] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:14:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:14:52.812340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16c01e-e989-440d-a1eb-5e9b2c2af098 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:52.812779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16c01e-e989-440d-a1eb-5e9b2c2af098 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:52.812871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16c01e-e989-440d-a1eb-5e9b2c2af098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:52.830554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1c16c01e-e989-440d-a1eb-5e9b2c2af098 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:14:52.831727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1c16c01e-e989-440d-a1eb-5e9b2c2af098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:14:52.895362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 636/1275] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:14:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:14:53.163803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb7191-c20e-43f8-973f-a836c59ed802 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:53.261326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb7191-c20e-43f8-973f-a836c59ed802 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:53.277433 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:53.293431 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bd3bd-07e4-4be2-b461-327979ad6638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:53.294035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bd3bd-07e4-4be2-b461-327979ad6638 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:53.294342 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bd3bd-07e4-4be2-b461-327979ad6638 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:53.294607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bd3bd-07e4-4be2-b461-327979ad6638 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:53.304882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bd3bd-07e4-4be2-b461-327979ad6638 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-899bd3bd-07e4-4be2-b461-327979ad6638 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 27 17:14:53.309595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:53.310754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:53.311793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:53.311793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:53.311793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:53.311793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:53.311793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 27 17:14:53.311793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 27 17:14:53.311793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 27 17:14:53.311793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 27 17:14:53.311793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 27 17:14:53.311793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 27 17:14:53.311793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 27 17:14:53.311793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:53.311793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 640/1276] 10.222.0.55 () {56 vars in 1096 bytes} [Tue Sep 27 17:14:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:53.317125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16c01e-e989-440d-a1eb-5e9b2c2af098 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:53.394215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:53.412457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16c01e-e989-440d-a1eb-5e9b2c2af098 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:53.445492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0db6d-8e2a-42b8-8b9a-2d212ff3df5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:53.446020 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0db6d-8e2a-42b8-8b9a-2d212ff3df5d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:53.446323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0db6d-8e2a-42b8-8b9a-2d212ff3df5d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:53.446600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0db6d-8e2a-42b8-8b9a-2d212ff3df5d None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:53.447253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0db6d-8e2a-42b8-8b9a-2d212ff3df5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:53.447553 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0db6d-8e2a-42b8-8b9a-2d212ff3df5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:53.448906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e0db6d-8e2a-42b8-8b9a-2d212ff3df5d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:53.456594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 637/1277] 10.222.0.55 () {56 vars in 1099 bytes} [Tue Sep 27 17:14:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:53.462344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bd3bd-07e4-4be2-b461-327979ad6638 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:53.558621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bd3bd-07e4-4be2-b461-327979ad6638 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:53.591624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437ec27-5186-4805-a270-6ea45d769492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:53.592239 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437ec27-5186-4805-a270-6ea45d769492 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:53.592514 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437ec27-5186-4805-a270-6ea45d769492 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:53.592782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437ec27-5186-4805-a270-6ea45d769492 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:53.602514 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437ec27-5186-4805-a270-6ea45d769492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:53.604181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437ec27-5186-4805-a270-6ea45d769492 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-5437ec27-5186-4805-a270-6ea45d769492 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:14:53.618427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:53.619634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:14:53.620745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:14:53.620745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:53.620745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:53.620745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 641/1278] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:53.625240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0db6d-8e2a-42b8-8b9a-2d212ff3df5d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:53.721702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0db6d-8e2a-42b8-8b9a-2d212ff3df5d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:53.753512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5272688-c025-4093-8a0d-90a295b8ac2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:53.754115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5272688-c025-4093-8a0d-90a295b8ac2b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:53.754394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5272688-c025-4093-8a0d-90a295b8ac2b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:53.754663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5272688-c025-4093-8a0d-90a295b8ac2b None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:53.755302 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5272688-c025-4093-8a0d-90a295b8ac2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:53.755602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5272688-c025-4093-8a0d-90a295b8ac2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:53.757457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5272688-c025-4093-8a0d-90a295b8ac2b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:53.768639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 638/1279] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:53.774907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437ec27-5186-4805-a270-6ea45d769492 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:53.804578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:53.870903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437ec27-5186-4805-a270-6ea45d769492 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:53.902850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b9ac1-dc73-4010-a2c0-c475ffa6fb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:53.903454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b9ac1-dc73-4010-a2c0-c475ffa6fb47 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:53.903729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b9ac1-dc73-4010-a2c0-c475ffa6fb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:53.903999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b9ac1-dc73-4010-a2c0-c475ffa6fb47 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:53.911010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b9ac1-dc73-4010-a2c0-c475ffa6fb47 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:53.912417 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b9ac1-dc73-4010-a2c0-c475ffa6fb47 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-a78b9ac1-dc73-4010-a2c0-c475ffa6fb47 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 27 17:14:53.923103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:53.924282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:53.925504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:53.925504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:53.925504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 27 17:14:53.925504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 27 17:14:53.925504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 27 17:14:53.925504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 27 17:14:53.925504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 27 17:14:53.925504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 27 17:14:53.925504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 27 17:14:53.925504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:53.925504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 642/1280] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:53.930551 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5272688-c025-4093-8a0d-90a295b8ac2b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:53.934325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:54.027944 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5272688-c025-4093-8a0d-90a295b8ac2b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:54.060300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5eaba9-0232-4045-98ba-29b693523678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:54.060889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eaba9-0232-4045-98ba-29b693523678 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:54.061167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eaba9-0232-4045-98ba-29b693523678 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:54.061438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eaba9-0232-4045-98ba-29b693523678 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:54.062183 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5eaba9-0232-4045-98ba-29b693523678 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:54.062488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5eaba9-0232-4045-98ba-29b693523678 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:54.063906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5eaba9-0232-4045-98ba-29b693523678 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:54.080719 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 639/1281] 10.222.0.55 () {56 vars in 1143 bytes} [Tue Sep 27 17:14:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:54.086831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b9ac1-dc73-4010-a2c0-c475ffa6fb47 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:54.174652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:54.184734 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b9ac1-dc73-4010-a2c0-c475ffa6fb47 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:54.216125 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d3ca4-6fcd-4212-8dbd-0f651df5e224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:54.216710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d3ca4-6fcd-4212-8dbd-0f651df5e224 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:54.216983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d3ca4-6fcd-4212-8dbd-0f651df5e224 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:54.217250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d3ca4-6fcd-4212-8dbd-0f651df5e224 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:54.281108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d3ca4-6fcd-4212-8dbd-0f651df5e224 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:54.282512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d3ca4-6fcd-4212-8dbd-0f651df5e224 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:54.367623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:54.377876 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 643/1282] 10.222.0.55 () {58 vars in 1282 bytes} [Tue Sep 27 17:14:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 27 17:14:54.383678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5eaba9-0232-4045-98ba-29b693523678 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:54.445219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:54.482770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5eaba9-0232-4045-98ba-29b693523678 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:54.516317 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf50b8-9b4f-4546-98c8-dae090970c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:54.516931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf50b8-9b4f-4546-98c8-dae090970c82 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:54.517209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf50b8-9b4f-4546-98c8-dae090970c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:54.517479 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf50b8-9b4f-4546-98c8-dae090970c82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:54.527701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf50b8-9b4f-4546-98c8-dae090970c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:54.529127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf50b8-9b4f-4546-98c8-dae090970c82 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a6cf50b8-9b4f-4546-98c8-dae090970c82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 27 17:14:54.543781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:54.544869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 27 17:14:54.545845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 27 17:14:54.545845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 27 17:14:54.545845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:54.545845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 640/1283] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:54.550562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d3ca4-6fcd-4212-8dbd-0f651df5e224 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:54.644633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d3ca4-6fcd-4212-8dbd-0f651df5e224 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:54.677267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcac04-34fb-4152-ba7d-93dd390158f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:54.677850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcac04-34fb-4152-ba7d-93dd390158f3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:54.678150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcac04-34fb-4152-ba7d-93dd390158f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:54.678420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bcac04-34fb-4152-ba7d-93dd390158f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:54.679093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcac04-34fb-4152-ba7d-93dd390158f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:54.679388 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcac04-34fb-4152-ba7d-93dd390158f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:54.680741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bcac04-34fb-4152-ba7d-93dd390158f3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:54.691761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 644/1284] 10.222.0.55 () {56 vars in 1109 bytes} [Tue Sep 27 17:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:54.697369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf50b8-9b4f-4546-98c8-dae090970c82 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:54.794503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf50b8-9b4f-4546-98c8-dae090970c82 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:54.807408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:54.826950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad6938-5a9c-472a-9428-e03fa8bbd426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:54.827558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad6938-5a9c-472a-9428-e03fa8bbd426 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:54.827834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad6938-5a9c-472a-9428-e03fa8bbd426 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:54.828101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad6938-5a9c-472a-9428-e03fa8bbd426 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:54.834516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad6938-5a9c-472a-9428-e03fa8bbd426 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:54.836477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad6938-5a9c-472a-9428-e03fa8bbd426 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-b6ad6938-5a9c-472a-9428-e03fa8bbd426 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 27 17:14:54.847402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:54.848441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:54.849378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 27 17:14:54.849378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 27 17:14:54.849378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 27 17:14:54.849378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 27 17:14:54.849378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 27 17:14:54.849378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 27 17:14:54.849378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 27 17:14:54.849378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 27 17:14:54.849378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 641/1285] 10.222.0.55 () {56 vars in 1111 bytes} [Tue Sep 27 17:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:54.855019 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcac04-34fb-4152-ba7d-93dd390158f3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:54.903425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:54.951480 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bcac04-34fb-4152-ba7d-93dd390158f3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:54.985390 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b94570-63a7-48a3-91df-0984402d18e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:54.985981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b94570-63a7-48a3-91df-0984402d18e1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:54.986292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b94570-63a7-48a3-91df-0984402d18e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:54.986565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b94570-63a7-48a3-91df-0984402d18e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:54.987245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b94570-63a7-48a3-91df-0984402d18e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:54.987545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b94570-63a7-48a3-91df-0984402d18e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:54.988944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b94570-63a7-48a3-91df-0984402d18e1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:55.001066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 645/1286] 10.222.0.55 () {56 vars in 1148 bytes} [Tue Sep 27 17:14:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:55.007102 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad6938-5a9c-472a-9428-e03fa8bbd426 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:55.101667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad6938-5a9c-472a-9428-e03fa8bbd426 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:55.133611 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f178f-55ed-4fa7-8139-5629d980502b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:55.134214 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f178f-55ed-4fa7-8139-5629d980502b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:55.134490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f178f-55ed-4fa7-8139-5629d980502b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:55.134756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f178f-55ed-4fa7-8139-5629d980502b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:55.135469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f178f-55ed-4fa7-8139-5629d980502b None admin] RBAC: Adding query filter params (role.id=1dad3f1944544ab78cc0d01f38569791, scope.project.id=bb1f2574a4344711a56722bca143ae6c, user.id=3b16c430b870424c8d366c9a2959003e) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:55.135775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f178f-55ed-4fa7-8139-5629d980502b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:55.137185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f178f-55ed-4fa7-8139-5629d980502b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:55.142686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 642/1287] 10.222.0.55 () {56 vars in 1374 bytes} [Tue Sep 27 17:14:55 2022] GET /identity/v3/role_assignments?user.id=3b16c430b870424c8d366c9a2959003e&scope.project.id=bb1f2574a4344711a56722bca143ae6c&role.id=1dad3f1944544ab78cc0d01f38569791 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:14:57.277925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c842ef-70ec-4f72-83c3-784e37249c3e None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:57.278490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c842ef-70ec-4f72-83c3-784e37249c3e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:57.278490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c842ef-70ec-4f72-83c3-784e37249c3e None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:57.279838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 646/1288] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:14:57.284853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1188e30-149e-44f2-a48e-fe70e345e4f1 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:57.285248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1188e30-149e-44f2-a48e-fe70e345e4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:57.285400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1188e30-149e-44f2-a48e-fe70e345e4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:57.304346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b1188e30-149e-44f2-a48e-fe70e345e4f1 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:14:57.305511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b1188e30-149e-44f2-a48e-fe70e345e4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:14:57.334269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:57.372447 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 643/1289] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:14:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:14:57.636506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c842ef-70ec-4f72-83c3-784e37249c3e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:57.734553 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c842ef-70ec-4f72-83c3-784e37249c3e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:57.745181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:14:57.767699 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eb0d5-e467-476b-98b2-8f54b5bd9124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:57.768309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eb0d5-e467-476b-98b2-8f54b5bd9124 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:57.768586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eb0d5-e467-476b-98b2-8f54b5bd9124 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:57.768858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eb0d5-e467-476b-98b2-8f54b5bd9124 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:57.769432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863eb0d5-e467-476b-98b2-8f54b5bd9124 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:57.770801 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863eb0d5-e467-476b-98b2-8f54b5bd9124 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-863eb0d5-e467-476b-98b2-8f54b5bd9124 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:14:57.779794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:14:57.780997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:14:57.782265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 647/1290] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Sep 27 17:14:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:14:57.786813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1188e30-149e-44f2-a48e-fe70e345e4f1 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:57.883079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1188e30-149e-44f2-a48e-fe70e345e4f1 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:57.915615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a36e50-c485-45d9-b3dc-f68704ad49d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:57.916147 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a36e50-c485-45d9-b3dc-f68704ad49d0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:57.916420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a36e50-c485-45d9-b3dc-f68704ad49d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:57.916688 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a36e50-c485-45d9-b3dc-f68704ad49d0 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:57.917307 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a36e50-c485-45d9-b3dc-f68704ad49d0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:57.917607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a36e50-c485-45d9-b3dc-f68704ad49d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:57.918993 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a36e50-c485-45d9-b3dc-f68704ad49d0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:57.923576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 644/1291] 10.222.0.55 () {56 vars in 1116 bytes} [Tue Sep 27 17:14:57 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:14:57.929193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eb0d5-e467-476b-98b2-8f54b5bd9124 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:14:58.041231 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eb0d5-e467-476b-98b2-8f54b5bd9124 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:14:58.073158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601fa9c-2c1b-4c2b-bb1c-5dfe6349ae5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:14:58.073751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601fa9c-2c1b-4c2b-bb1c-5dfe6349ae5d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:14:58.074025 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601fa9c-2c1b-4c2b-bb1c-5dfe6349ae5d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:14:58.074333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601fa9c-2c1b-4c2b-bb1c-5dfe6349ae5d None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:14:58.074958 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601fa9c-2c1b-4c2b-bb1c-5dfe6349ae5d None admin] RBAC: Adding query filter params (type=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:14:58.075254 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601fa9c-2c1b-4c2b-bb1c-5dfe6349ae5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:14:58.076602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601fa9c-2c1b-4c2b-bb1c-5dfe6349ae5d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:14:58.080799 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 648/1292] 10.222.0.55 () {56 vars in 1116 bytes} [Tue Sep 27 17:14:57 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:15:00.235204 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a5ac6-dd93-4664-a109-d4cc3226521c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:00.235928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a5ac6-dd93-4664-a109-d4cc3226521c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:00.235928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a5ac6-dd93-4664-a109-d4cc3226521c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:00.237062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 645/1293] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:15:00.242251 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea367466-68ff-418f-8673-74e774ed5d37 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:00.242695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea367466-68ff-418f-8673-74e774ed5d37 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:00.242787 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea367466-68ff-418f-8673-74e774ed5d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:00.260953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ea367466-68ff-418f-8673-74e774ed5d37 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:15:00.262152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ea367466-68ff-418f-8673-74e774ed5d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:15:00.291338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:15:00.328250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 649/1294] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:15:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:15:00.597792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a5ac6-dd93-4664-a109-d4cc3226521c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:15:00.628338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:15:00.696428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a5ac6-dd93-4664-a109-d4cc3226521c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:15:00.729958 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c345e5-1b43-4079-9af8-2e973c7b4297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:15:00.730574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c345e5-1b43-4079-9af8-2e973c7b4297 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:00.730854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c345e5-1b43-4079-9af8-2e973c7b4297 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:00.731129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c345e5-1b43-4079-9af8-2e973c7b4297 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:00.731778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c345e5-1b43-4079-9af8-2e973c7b4297 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:15:00.733142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c345e5-1b43-4079-9af8-2e973c7b4297 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:15:00.748260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 646/1295] 10.222.0.55 () {60 vars in 1141 bytes} [Tue Sep 27 17:15:00 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:15:02.857423 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e963ac5-3205-4ba5-81cc-a045ab23fe10 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:02.857967 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e963ac5-3205-4ba5-81cc-a045ab23fe10 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:02.857967 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e963ac5-3205-4ba5-81cc-a045ab23fe10 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:02.859338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 650/1296] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:15:02.864403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed30f9-63c8-4115-b7cd-778f34a596e3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:02.864800 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed30f9-63c8-4115-b7cd-778f34a596e3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:02.864946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed30f9-63c8-4115-b7cd-778f34a596e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:02.883700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a0ed30f9-63c8-4115-b7cd-778f34a596e3 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:15:02.884874 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a0ed30f9-63c8-4115-b7cd-778f34a596e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:15:02.954306 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 647/1297] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:15:02 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:15:03.238619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e963ac5-3205-4ba5-81cc-a045ab23fe10 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:15:03.335985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e963ac5-3205-4ba5-81cc-a045ab23fe10 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:15:03.369461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b9c3e-60f9-4870-b7a7-3578d5289689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:15:03.370092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b9c3e-60f9-4870-b7a7-3578d5289689 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:03.370369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b9c3e-60f9-4870-b7a7-3578d5289689 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:03.370637 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b9c3e-60f9-4870-b7a7-3578d5289689 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:03.371208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b9c3e-60f9-4870-b7a7-3578d5289689 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:15:03.372548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b9c3e-60f9-4870-b7a7-3578d5289689 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-959b9c3e-60f9-4870-b7a7-3578d5289689 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:15:03.381004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:15:03.382220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:15:03.383438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 651/1298] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Sep 27 17:15:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:15:03.388094 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed30f9-63c8-4115-b7cd-778f34a596e3 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:15:03.405640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:15:03.484888 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed30f9-63c8-4115-b7cd-778f34a596e3 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:15:03.517671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d30ad0-b5de-4ee3-bdc4-2993252c7aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:15:03.518267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d30ad0-b5de-4ee3-bdc4-2993252c7aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:03.518547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d30ad0-b5de-4ee3-bdc4-2993252c7aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:03.518819 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d30ad0-b5de-4ee3-bdc4-2993252c7aa3 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:03.519448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d30ad0-b5de-4ee3-bdc4-2993252c7aa3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:15:03.519750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d30ad0-b5de-4ee3-bdc4-2993252c7aa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:15:03.521098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d30ad0-b5de-4ee3-bdc4-2993252c7aa3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:15:03.526678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 648/1299] 10.222.0.55 () {56 vars in 1116 bytes} [Tue Sep 27 17:15:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:15:03.532652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b9c3e-60f9-4870-b7a7-3578d5289689 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:15:03.563569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:15:03.631252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b9c3e-60f9-4870-b7a7-3578d5289689 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:15:03.664218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f9d15-94a6-4a76-ae1c-00bf697ed31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:15:03.664799 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f9d15-94a6-4a76-ae1c-00bf697ed31e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:03.665075 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f9d15-94a6-4a76-ae1c-00bf697ed31e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:03.665346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f9d15-94a6-4a76-ae1c-00bf697ed31e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:03.666034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f9d15-94a6-4a76-ae1c-00bf697ed31e None admin] RBAC: Adding query filter params (interface=public, service_id=d93130d1cc064632a42f2457c92f5d68, region_id=RegionOne) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:15:03.666367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f9d15-94a6-4a76-ae1c-00bf697ed31e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:15:03.667708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f9d15-94a6-4a76-ae1c-00bf697ed31e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:15:03.672544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 652/1300] 10.222.0.55 () {56 vars in 1251 bytes} [Tue Sep 27 17:15:03 2022] GET /identity/v3/endpoints?service_id=d93130d1cc064632a42f2457c92f5d68&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:15:03.678216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d30ad0-b5de-4ee3-bdc4-2993252c7aa3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:15:03.756045 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:15:03.773956 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d30ad0-b5de-4ee3-bdc4-2993252c7aa3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:15:03.808485 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c4618-fc44-4604-8f07-7457f06c213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:15:03.809089 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c4618-fc44-4604-8f07-7457f06c213b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:03.809360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c4618-fc44-4604-8f07-7457f06c213b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:03.811029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c4618-fc44-4604-8f07-7457f06c213b None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:03.811029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c4618-fc44-4604-8f07-7457f06c213b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:15:03.811769 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c4618-fc44-4604-8f07-7457f06c213b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:15:03.819135 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 649/1301] 10.222.0.55 () {56 vars in 1087 bytes} [Tue Sep 27 17:15:03 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:15:05.949785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb15fd8-b6ca-49b6-a79e-32b387482e9d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:05.950454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb15fd8-b6ca-49b6-a79e-32b387482e9d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:05.950454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb15fd8-b6ca-49b6-a79e-32b387482e9d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:05.951656 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 653/1302] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:15:05.956866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd368e-08b4-43b0-a820-d2ea962d4ec5 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:05.957284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd368e-08b4-43b0-a820-d2ea962d4ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:05.957435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd368e-08b4-43b0-a820-d2ea962d4ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:05.976225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f4dd368e-08b4-43b0-a820-d2ea962d4ec5 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:15:05.977396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f4dd368e-08b4-43b0-a820-d2ea962d4ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:15:06.040698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 650/1303] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:15:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 27 17:15:06.312760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb15fd8-b6ca-49b6-a79e-32b387482e9d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:15:06.405706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:15:06.412694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb15fd8-b6ca-49b6-a79e-32b387482e9d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:15:06.444996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac19e6c-9865-4739-b963-648547d24bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:15:06.445604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac19e6c-9865-4739-b963-648547d24bcc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:06.445881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac19e6c-9865-4739-b963-648547d24bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:06.446178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac19e6c-9865-4739-b963-648547d24bcc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:06.446750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac19e6c-9865-4739-b963-648547d24bcc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:15:06.448091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac19e6c-9865-4739-b963-648547d24bcc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-3ac19e6c-9865-4739-b963-648547d24bcc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 27 17:15:06.456531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 27 17:15:06.457713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 27 17:15:06.458774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 654/1304] 10.222.0.55 () {56 vars in 1117 bytes} [Tue Sep 27 17:15:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 27 17:15:06.464206 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd368e-08b4-43b0-a820-d2ea962d4ec5 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:15:06.537496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:15:06.560036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd368e-08b4-43b0-a820-d2ea962d4ec5 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:15:06.601511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb970e-f9ac-434f-a081-f1927b8bd6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:15:06.602138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb970e-f9ac-434f-a081-f1927b8bd6ba None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:06.602417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb970e-f9ac-434f-a081-f1927b8bd6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:06.602689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb970e-f9ac-434f-a081-f1927b8bd6ba None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:06.603570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb970e-f9ac-434f-a081-f1927b8bd6ba None admin] RBAC: Adding query filter params (name=placement) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:15:06.603870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb970e-f9ac-434f-a081-f1927b8bd6ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:15:06.605231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bb970e-f9ac-434f-a081-f1927b8bd6ba None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:15:06.610243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 651/1305] 10.222.0.55 () {56 vars in 1116 bytes} [Tue Sep 27 17:15:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:15:06.616702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac19e6c-9865-4739-b963-648547d24bcc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:15:06.713100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac19e6c-9865-4739-b963-648547d24bcc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:15:06.745707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef74eec-9557-4dae-9d31-a72d425b04e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:15:06.746313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef74eec-9557-4dae-9d31-a72d425b04e2 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:15:06.746586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef74eec-9557-4dae-9d31-a72d425b04e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:15:06.746851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef74eec-9557-4dae-9d31-a72d425b04e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:15:06.747495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef74eec-9557-4dae-9d31-a72d425b04e2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:15:06.748835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef74eec-9557-4dae-9d31-a72d425b04e2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:15:06.767668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 655/1306] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:15:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:15:16.607355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=30 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:15:16.763497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:10.675602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742218de-ac9e-4654-9217-196aad0f072c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:10.676140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742218de-ac9e-4654-9217-196aad0f072c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:10.676230 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742218de-ac9e-4654-9217-196aad0f072c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:10.681523 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 652/1307] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:16:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:10.682946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bdcd3-367d-43fb-888c-9cda0f3d1efd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:10.683343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bdcd3-367d-43fb-888c-9cda0f3d1efd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:10.683577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bdcd3-367d-43fb-888c-9cda0f3d1efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:10.809642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8b5bdcd3-367d-43fb-888c-9cda0f3d1efd None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:10.810986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8b5bdcd3-367d-43fb-888c-9cda0f3d1efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:10.935084 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 656/1308] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:16:11.234029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79617d3-f360-42f5-aaf6-1abdae12e347 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:11.234417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79617d3-f360-42f5-aaf6-1abdae12e347 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:11.234621 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79617d3-f360-42f5-aaf6-1abdae12e347 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:11.236007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 653/1309] 10.222.0.55 () {54 vars in 964 bytes} [Tue Sep 27 17:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:11.243032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25459dbc-0c58-402a-8fbe-6743e3e7d40e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:11.243617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25459dbc-0c58-402a-8fbe-6743e3e7d40e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:11.244161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25459dbc-0c58-402a-8fbe-6743e3e7d40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:11.334647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-25459dbc-0c58-402a-8fbe-6743e3e7d40e None None] MFA Rules not processed for user `5d906452199844149966c9c86db4684e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:11.335873 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-25459dbc-0c58-402a-8fbe-6743e3e7d40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:11.357411 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:11.417255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 657/1310] 10.222.0.55 () {58 vars in 1061 bytes} [Tue Sep 27 17:16:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:16:11.426771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79617d3-f360-42f5-aaf6-1abdae12e347 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:16:11.428509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d79617d3-f360-42f5-aaf6-1abdae12e347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:11.547596 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79617d3-f360-42f5-aaf6-1abdae12e347 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:16:11.634015 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d81650-3b1e-4679-8791-363fc36c89f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:16:11.634764 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d81650-3b1e-4679-8791-363fc36c89f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:11.635114 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d81650-3b1e-4679-8791-363fc36c89f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:11.635450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d81650-3b1e-4679-8791-363fc36c89f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:11.637160 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-84d81650-3b1e-4679-8791-363fc36c89f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:11.701383 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:11.763418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d81650-3b1e-4679-8791-363fc36c89f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:16:11.764918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d81650-3b1e-4679-8791-363fc36c89f7 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:16:11.785063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 654/1311] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:16:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:16:14.327881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82574671-375e-45b0-81ab-fbfc7232729f None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:14.328301 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82574671-375e-45b0-81ab-fbfc7232729f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:14.328422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82574671-375e-45b0-81ab-fbfc7232729f None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:14.329764 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 658/1312] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:14.335072 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4b9aa-3ada-41a9-9dfb-dacca9dad05c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:14.335470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4b9aa-3ada-41a9-9dfb-dacca9dad05c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:14.335609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4b9aa-3ada-41a9-9dfb-dacca9dad05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:14.408915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-02b4b9aa-3ada-41a9-9dfb-dacca9dad05c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:14.410683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-02b4b9aa-3ada-41a9-9dfb-dacca9dad05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:14.504250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 655/1313] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:16:14.805313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f695df-328f-492b-9de4-5043c1499ebb None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:14.805586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f695df-328f-492b-9de4-5043c1499ebb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:14.805855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f695df-328f-492b-9de4-5043c1499ebb None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:14.807236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 659/1314] 10.222.0.55 () {54 vars in 964 bytes} [Tue Sep 27 17:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:14.812921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e3626-a114-461f-93a2-9b5cb44f2f8e None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:14.813200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e3626-a114-461f-93a2-9b5cb44f2f8e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:14.813467 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e3626-a114-461f-93a2-9b5cb44f2f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:14.827501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:14.896092 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-035e3626-a114-461f-93a2-9b5cb44f2f8e None None] MFA Rules not processed for user `5d906452199844149966c9c86db4684e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:14.897280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-035e3626-a114-461f-93a2-9b5cb44f2f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:14.970934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 656/1315] 10.222.0.55 () {58 vars in 1061 bytes} [Tue Sep 27 17:16:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:16:14.980206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f695df-328f-492b-9de4-5043c1499ebb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:16:14.981780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-80f695df-328f-492b-9de4-5043c1499ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:15.085778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f695df-328f-492b-9de4-5043c1499ebb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:16:15.155633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:15.157996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab153ae7-419a-424b-a46e-1be8ce7aa7e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:16:15.158791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab153ae7-419a-424b-a46e-1be8ce7aa7e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:15.159126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab153ae7-419a-424b-a46e-1be8ce7aa7e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:15.159596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab153ae7-419a-424b-a46e-1be8ce7aa7e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:15.161912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ab153ae7-419a-424b-a46e-1be8ce7aa7e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:15.271915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab153ae7-419a-424b-a46e-1be8ce7aa7e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:16:15.273329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab153ae7-419a-424b-a46e-1be8ce7aa7e7 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:16:15.290800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 660/1316] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:16:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:16:17.981374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b3fad-8b0f-447f-9712-8db5db30ca0d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:17.981910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b3fad-8b0f-447f-9712-8db5db30ca0d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:17.982034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b3fad-8b0f-447f-9712-8db5db30ca0d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:17.983311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 657/1317] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:17.989095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e30ff0-0c4e-418b-bbfd-a4aea9eee914 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:17.989493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e30ff0-0c4e-418b-bbfd-a4aea9eee914 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:17.989627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e30ff0-0c4e-418b-bbfd-a4aea9eee914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:18.053794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-52e30ff0-0c4e-418b-bbfd-a4aea9eee914 None None] MFA Rules not processed for user `c8d29301d2f84fcdb72d19727075628d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:18.055283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-52e30ff0-0c4e-418b-bbfd-a4aea9eee914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:18.106260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:18.150861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 661/1318] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:16:17 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:16:18.440738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b3fad-8b0f-447f-9712-8db5db30ca0d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:16:18.442306 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-569b3fad-8b0f-447f-9712-8db5db30ca0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:18.544065 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b3fad-8b0f-447f-9712-8db5db30ca0d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:16:18.546852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:18.616357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9599006-f804-43c7-9dc2-4a64e103e165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:16:18.616949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9599006-f804-43c7-9dc2-4a64e103e165 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:18.617224 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9599006-f804-43c7-9dc2-4a64e103e165 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:18.617490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9599006-f804-43c7-9dc2-4a64e103e165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:18.619084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f9599006-f804-43c7-9dc2-4a64e103e165 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:18.723756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9599006-f804-43c7-9dc2-4a64e103e165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:16:18.725166 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9599006-f804-43c7-9dc2-4a64e103e165 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:16:18.743964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 658/1319] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:16:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:16:21.449274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72e6a7-2437-4e18-bf07-c763f78fc320 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:21.449827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72e6a7-2437-4e18-bf07-c763f78fc320 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:21.449827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72e6a7-2437-4e18-bf07-c763f78fc320 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:21.451203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 662/1320] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:21.456450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e7f8-ad63-4ddb-b4c0-182ff713bf0b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:21.456827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e7f8-ad63-4ddb-b4c0-182ff713bf0b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:21.456993 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e7f8-ad63-4ddb-b4c0-182ff713bf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:21.501204 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:21.526617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2ee0e7f8-ad63-4ddb-b4c0-182ff713bf0b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:21.527821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2ee0e7f8-ad63-4ddb-b4c0-182ff713bf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:21.625748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 659/1321] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:16:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:16:21.916953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72e6a7-2437-4e18-bf07-c763f78fc320 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:16:21.918553 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ed72e6a7-2437-4e18-bf07-c763f78fc320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:22.021244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72e6a7-2437-4e18-bf07-c763f78fc320 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:16:22.093007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de164158-4ca0-406c-9268-ffb719e09817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:16:22.093599 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de164158-4ca0-406c-9268-ffb719e09817 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:22.093875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de164158-4ca0-406c-9268-ffb719e09817 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:22.094181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de164158-4ca0-406c-9268-ffb719e09817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:22.095769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-de164158-4ca0-406c-9268-ffb719e09817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:22.103357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:22.203506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de164158-4ca0-406c-9268-ffb719e09817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:16:22.204852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de164158-4ca0-406c-9268-ffb719e09817 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:16:22.222645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 663/1322] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:16:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:16:25.019463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae01710-0f07-4e75-a27e-fdef1d04d623 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:25.020013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae01710-0f07-4e75-a27e-fdef1d04d623 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:25.020013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae01710-0f07-4e75-a27e-fdef1d04d623 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:25.021351 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 660/1323] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:25.026612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623272a0-668e-4190-95be-8a35ad353852 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:25.027041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623272a0-668e-4190-95be-8a35ad353852 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:25.027143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623272a0-668e-4190-95be-8a35ad353852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:25.100704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-623272a0-668e-4190-95be-8a35ad353852 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:25.102006 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-623272a0-668e-4190-95be-8a35ad353852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:25.164707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:25.206997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 664/1324] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:16:25.489996 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae01710-0f07-4e75-a27e-fdef1d04d623 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:16:25.491639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2ae01710-0f07-4e75-a27e-fdef1d04d623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:25.594336 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae01710-0f07-4e75-a27e-fdef1d04d623 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:16:25.607201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:25.667237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19662d57-65cf-413b-a560-413b75872fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:16:25.667833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19662d57-65cf-413b-a560-413b75872fcd service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:25.668108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19662d57-65cf-413b-a560-413b75872fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:25.668371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19662d57-65cf-413b-a560-413b75872fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:25.669981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-19662d57-65cf-413b-a560-413b75872fcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:25.777580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19662d57-65cf-413b-a560-413b75872fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:16:25.779047 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19662d57-65cf-413b-a560-413b75872fcd service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:16:25.797253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 661/1325] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:16:28.010156 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02795e28-7399-49ed-8432-ab4d5127c501 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:28.010688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02795e28-7399-49ed-8432-ab4d5127c501 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:28.010688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02795e28-7399-49ed-8432-ab4d5127c501 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:28.012076 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 665/1326] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:28.017477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e10d3-314f-472d-b3af-dd19ea75e84e None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:28.017920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e10d3-314f-472d-b3af-dd19ea75e84e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:28.018016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e10d3-314f-472d-b3af-dd19ea75e84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:28.068380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:28.087329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-244e10d3-314f-472d-b3af-dd19ea75e84e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:28.088631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-244e10d3-314f-472d-b3af-dd19ea75e84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:28.193464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 662/1327] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:16:28.474277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02795e28-7399-49ed-8432-ab4d5127c501 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:16:28.475906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-02795e28-7399-49ed-8432-ab4d5127c501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:28.575590 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02795e28-7399-49ed-8432-ab4d5127c501 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:16:28.650018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c56f0a-dbdf-4ceb-9203-67e6a4a1cc3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:16:28.650648 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c56f0a-dbdf-4ceb-9203-67e6a4a1cc3b service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:28.650925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c56f0a-dbdf-4ceb-9203-67e6a4a1cc3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:28.651193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c56f0a-dbdf-4ceb-9203-67e6a4a1cc3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:28.652789 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f6c56f0a-dbdf-4ceb-9203-67e6a4a1cc3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:28.669958 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:28.763445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c56f0a-dbdf-4ceb-9203-67e6a4a1cc3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:16:28.764780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c56f0a-dbdf-4ceb-9203-67e6a4a1cc3b service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:16:28.783536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 666/1328] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:16:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:16:31.029639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427e7eb-2267-4b8b-96ca-772866b9f449 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:31.030164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427e7eb-2267-4b8b-96ca-772866b9f449 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:31.030164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427e7eb-2267-4b8b-96ca-772866b9f449 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:31.031507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 663/1329] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:31.036859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbac99-27dd-4283-b3b7-69de15193787 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:31.037268 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbac99-27dd-4283-b3b7-69de15193787 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:31.037400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbac99-27dd-4283-b3b7-69de15193787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:31.107854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-85fbac99-27dd-4283-b3b7-69de15193787 None None] MFA Rules not processed for user `c8d29301d2f84fcdb72d19727075628d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:31.109072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-85fbac99-27dd-4283-b3b7-69de15193787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:31.172483 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:31.205438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 667/1330] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:16:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:16:31.495096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427e7eb-2267-4b8b-96ca-772866b9f449 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:16:31.496613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4427e7eb-2267-4b8b-96ca-772866b9f449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:31.604029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427e7eb-2267-4b8b-96ca-772866b9f449 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:16:31.625791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:31.687592 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4c508-2dff-4abd-b548-c666801ec4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:16:31.688195 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4c508-2dff-4abd-b548-c666801ec4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:31.688470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4c508-2dff-4abd-b548-c666801ec4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:31.688734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4c508-2dff-4abd-b548-c666801ec4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:31.690377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1cd4c508-2dff-4abd-b548-c666801ec4c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:31.823344 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4c508-2dff-4abd-b548-c666801ec4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:16:31.824700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4c508-2dff-4abd-b548-c666801ec4c3 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:16:31.847607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 664/1331] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:16:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:16:34.263131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810f3e5-0179-4f93-b2d4-b6b53d02aa18 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:34.263632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810f3e5-0179-4f93-b2d4-b6b53d02aa18 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:34.263724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810f3e5-0179-4f93-b2d4-b6b53d02aa18 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:34.265058 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 668/1332] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:34.271167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442804b-ded5-4a20-824f-5e7711b10d2d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:34.271630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442804b-ded5-4a20-824f-5e7711b10d2d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:34.271878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442804b-ded5-4a20-824f-5e7711b10d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:34.337027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:34.340152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f442804b-ded5-4a20-824f-5e7711b10d2d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:34.341827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f442804b-ded5-4a20-824f-5e7711b10d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:34.439465 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 665/1333] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:16:34.722040 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810f3e5-0179-4f93-b2d4-b6b53d02aa18 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:16:34.723617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b810f3e5-0179-4f93-b2d4-b6b53d02aa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:34.824937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810f3e5-0179-4f93-b2d4-b6b53d02aa18 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:16:34.897994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fea5c-f7f1-4b6e-bf81-e9b5722a92a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:16:34.898606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fea5c-f7f1-4b6e-bf81-e9b5722a92a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:34.898880 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fea5c-f7f1-4b6e-bf81-e9b5722a92a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:34.899149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fea5c-f7f1-4b6e-bf81-e9b5722a92a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:34.900731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5c5fea5c-f7f1-4b6e-bf81-e9b5722a92a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:34.926604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:35.007608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fea5c-f7f1-4b6e-bf81-e9b5722a92a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:16:35.008983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fea5c-f7f1-4b6e-bf81-e9b5722a92a7 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:16:35.027525 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 669/1334] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:16:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:16:37.199797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c1f28-bd85-4962-b784-2a4be650f244 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:37.200341 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c1f28-bd85-4962-b784-2a4be650f244 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:37.200341 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c1f28-bd85-4962-b784-2a4be650f244 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:37.201681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 666/1335] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:37.206861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4966be-abe7-4ea6-a1c3-2a8bf4a8fa6c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:37.207260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4966be-abe7-4ea6-a1c3-2a8bf4a8fa6c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:37.207403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4966be-abe7-4ea6-a1c3-2a8bf4a8fa6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:37.273937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3e4966be-abe7-4ea6-a1c3-2a8bf4a8fa6c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:37.275284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3e4966be-abe7-4ea6-a1c3-2a8bf4a8fa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:37.353025 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:37.376494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 670/1336] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:16:37.660187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c1f28-bd85-4962-b784-2a4be650f244 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:16:37.661692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-359c1f28-bd85-4962-b784-2a4be650f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:37.761651 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c1f28-bd85-4962-b784-2a4be650f244 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:16:37.789170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:37.833146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eba2ae-29ec-4a19-82ac-4333638a77b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:16:37.833750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba2ae-29ec-4a19-82ac-4333638a77b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:37.834029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba2ae-29ec-4a19-82ac-4333638a77b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:37.834331 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eba2ae-29ec-4a19-82ac-4333638a77b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:37.835930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f3eba2ae-29ec-4a19-82ac-4333638a77b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:37.941975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eba2ae-29ec-4a19-82ac-4333638a77b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:16:37.943364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eba2ae-29ec-4a19-82ac-4333638a77b6 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:16:37.961681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 667/1337] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:16:40.485040 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa1f95-e718-4e69-8978-201541f64980 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:40.485544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa1f95-e718-4e69-8978-201541f64980 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:40.485544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa1f95-e718-4e69-8978-201541f64980 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:40.486863 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 671/1338] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:40.492175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb501e-bc6d-4a87-afb8-312efa2d7642 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:40.492619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb501e-bc6d-4a87-afb8-312efa2d7642 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:40.492838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb501e-bc6d-4a87-afb8-312efa2d7642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:40.569070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c0bb501e-bc6d-4a87-afb8-312efa2d7642 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:40.570498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c0bb501e-bc6d-4a87-afb8-312efa2d7642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:40.579877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:40.681187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 668/1339] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:16:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:16:40.962201 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa1f95-e718-4e69-8978-201541f64980 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:16:40.963765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c8aa1f95-e718-4e69-8978-201541f64980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:41.069798 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa1f95-e718-4e69-8978-201541f64980 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:16:41.141071 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1769cb-1c4b-4b58-a1e3-ff12a37025ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:16:41.141661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1769cb-1c4b-4b58-a1e3-ff12a37025ae service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:41.141930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1769cb-1c4b-4b58-a1e3-ff12a37025ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:41.142227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1769cb-1c4b-4b58-a1e3-ff12a37025ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:41.143878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ac1769cb-1c4b-4b58-a1e3-ff12a37025ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:41.179360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:41.250713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1769cb-1c4b-4b58-a1e3-ff12a37025ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:16:41.252124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1769cb-1c4b-4b58-a1e3-ff12a37025ae service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:16:41.270015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 672/1340] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:16:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:16:45.317748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3bdd5-af55-48fe-ac5e-c3572cebe1fc None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:45.317748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3bdd5-af55-48fe-ac5e-c3572cebe1fc None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:45.317748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3bdd5-af55-48fe-ac5e-c3572cebe1fc None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:45.322230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916f10-5a0d-448b-a17c-7dfbcdad310d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:45.322663 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916f10-5a0d-448b-a17c-7dfbcdad310d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:45.322867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916f10-5a0d-448b-a17c-7dfbcdad310d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:45.326291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 669/1341] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:16:45 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:45.389645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fc916f10-5a0d-448b-a17c-7dfbcdad310d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:45.390867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fc916f10-5a0d-448b-a17c-7dfbcdad310d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:45.485641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:45.501230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 673/1342] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:16:45.788061 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3bdd5-af55-48fe-ac5e-c3572cebe1fc None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:16:45.789580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6ce3bdd5-af55-48fe-ac5e-c3572cebe1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:45.892732 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3bdd5-af55-48fe-ac5e-c3572cebe1fc None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:16:45.930033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:45.964901 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06d14e-cd12-4e77-9954-a35f813ab1f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:16:45.965502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06d14e-cd12-4e77-9954-a35f813ab1f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:45.965773 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06d14e-cd12-4e77-9954-a35f813ab1f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:45.966040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06d14e-cd12-4e77-9954-a35f813ab1f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:45.967667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-dd06d14e-cd12-4e77-9954-a35f813ab1f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:46.075463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06d14e-cd12-4e77-9954-a35f813ab1f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:16:46.076802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06d14e-cd12-4e77-9954-a35f813ab1f9 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:16:46.094786 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 670/1343] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:16:49.240942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969abda-4fbb-4184-9996-c0216cafb5d7 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:49.241472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969abda-4fbb-4184-9996-c0216cafb5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:49.241472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969abda-4fbb-4184-9996-c0216cafb5d7 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:49.242918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 674/1344] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:49.248235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae937ba-2865-4dcc-bb42-ca87ce6cc99d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:49.248640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae937ba-2865-4dcc-bb42-ca87ce6cc99d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:49.248898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae937ba-2865-4dcc-bb42-ca87ce6cc99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:49.316117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5ae937ba-2865-4dcc-bb42-ca87ce6cc99d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:49.317396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5ae937ba-2865-4dcc-bb42-ca87ce6cc99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:49.336244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:49.418231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 671/1345] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:16:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:16:49.699626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969abda-4fbb-4184-9996-c0216cafb5d7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:16:49.701162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6969abda-4fbb-4184-9996-c0216cafb5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:49.803702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969abda-4fbb-4184-9996-c0216cafb5d7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:16:49.904543 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771fd70-f142-486c-ba32-6e0b62ee7497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:16:49.905152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771fd70-f142-486c-ba32-6e0b62ee7497 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:49.905427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771fd70-f142-486c-ba32-6e0b62ee7497 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:49.905702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771fd70-f142-486c-ba32-6e0b62ee7497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:49.907359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8771fd70-f142-486c-ba32-6e0b62ee7497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:49.954941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:50.036075 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771fd70-f142-486c-ba32-6e0b62ee7497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:16:50.037511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771fd70-f142-486c-ba32-6e0b62ee7497 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:16:50.056030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 675/1346] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:16:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:16:54.406027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda819d6-e987-4cba-be1f-34e48fdf88a6 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:54.406585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda819d6-e987-4cba-be1f-34e48fdf88a6 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:54.406704 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda819d6-e987-4cba-be1f-34e48fdf88a6 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:54.408148 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 672/1347] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:16:54.413537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2bd37-6ca6-4472-ad42-5abb349a30bb None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:54.413965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2bd37-6ca6-4472-ad42-5abb349a30bb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:54.414111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2bd37-6ca6-4472-ad42-5abb349a30bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:54.485956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-49a2bd37-6ca6-4472-ad42-5abb349a30bb None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:16:54.487289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-49a2bd37-6ca6-4472-ad42-5abb349a30bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:54.587614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:54.590386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 676/1348] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:16:54.881111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda819d6-e987-4cba-be1f-34e48fdf88a6 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:16:54.882662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fda819d6-e987-4cba-be1f-34e48fdf88a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:54.984957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda819d6-e987-4cba-be1f-34e48fdf88a6 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:16:55.037167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:16:55.063937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35276e36-ab6a-4591-8b9f-24b4818ad8a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:16:55.064536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35276e36-ab6a-4591-8b9f-24b4818ad8a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:16:55.064809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35276e36-ab6a-4591-8b9f-24b4818ad8a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:16:55.065074 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35276e36-ab6a-4591-8b9f-24b4818ad8a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:16:55.066690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-35276e36-ab6a-4591-8b9f-24b4818ad8a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:16:55.172106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35276e36-ab6a-4591-8b9f-24b4818ad8a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:16:55.173450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35276e36-ab6a-4591-8b9f-24b4818ad8a8 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:16:55.191561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 673/1349] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:00.130810 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243733e-dd56-48b3-b603-136222bc6c14 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:00.131254 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243733e-dd56-48b3-b603-136222bc6c14 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:00.131521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243733e-dd56-48b3-b603-136222bc6c14 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:00.133410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 677/1350] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:00.138705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e688da-9309-4e52-8754-d734691fdc3c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:00.139137 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e688da-9309-4e52-8754-d734691fdc3c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:00.139236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e688da-9309-4e52-8754-d734691fdc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:00.280609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-40e688da-9309-4e52-8754-d734691fdc3c None None] MFA Rules not processed for user `3b16c430b870424c8d366c9a2959003e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:00.281790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-40e688da-9309-4e52-8754-d734691fdc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:00.306408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:00.412126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 674/1351] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:17:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:04.456644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a452b44-e1f7-48f2-b6c3-ddda1f5d2db3 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:04.457467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a452b44-e1f7-48f2-b6c3-ddda1f5d2db3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:04.458051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a452b44-e1f7-48f2-b6c3-ddda1f5d2db3 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:04.459532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 678/1352] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:17:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:04.465939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d183c-8f28-457e-b203-5e7808bb7665 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:04.466418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d183c-8f28-457e-b203-5e7808bb7665 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:04.466499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d183c-8f28-457e-b203-5e7808bb7665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:04.545624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-593d183c-8f28-457e-b203-5e7808bb7665 None None] MFA Rules not processed for user `3b16c430b870424c8d366c9a2959003e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:04.546906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-593d183c-8f28-457e-b203-5e7808bb7665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:04.626441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:04.665615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 675/1353] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:17:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:06.145387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7ca53-d92b-4076-81f4-051f65c75b2f None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:06.145901 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7ca53-d92b-4076-81f4-051f65c75b2f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:06.146096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7ca53-d92b-4076-81f4-051f65c75b2f None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:06.147513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 679/1354] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:06.152550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8aef4-dc0b-4fa2-924b-86068eacb6ee None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:06.152957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8aef4-dc0b-4fa2-924b-86068eacb6ee None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:06.153210 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8aef4-dc0b-4fa2-924b-86068eacb6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:06.229389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b8e8aef4-dc0b-4fa2-924b-86068eacb6ee None None] MFA Rules not processed for user `3b16c430b870424c8d366c9a2959003e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:06.230617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b8e8aef4-dc0b-4fa2-924b-86068eacb6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:06.343188 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 676/1355] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:17:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:09.881542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c771bd8-f8a7-4e4a-a960-141090ac8870 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:09.882100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c771bd8-f8a7-4e4a-a960-141090ac8870 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:09.882227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c771bd8-f8a7-4e4a-a960-141090ac8870 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:09.883779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 680/1356] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:09.889169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a11cf8-9c44-4ec2-8642-303b3b34074a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:09.889648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a11cf8-9c44-4ec2-8642-303b3b34074a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:09.889748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a11cf8-9c44-4ec2-8642-303b3b34074a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:09.929199 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:09.962330 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-39a11cf8-9c44-4ec2-8642-303b3b34074a None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:09.963448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-39a11cf8-9c44-4ec2-8642-303b3b34074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:10.065799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 677/1357] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:10.369301 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7bbdc-3e0c-4290-a70b-475abdb94131 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:10.370397 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7bbdc-3e0c-4290-a70b-475abdb94131 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:10.370879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7bbdc-3e0c-4290-a70b-475abdb94131 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:10.372945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 681/1358] 10.222.0.55 () {54 vars in 961 bytes} [Tue Sep 27 17:17:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:10.382508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaf24f-ee24-4b84-af01-6ff32fddde46 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:10.383269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaf24f-ee24-4b84-af01-6ff32fddde46 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:10.383808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaf24f-ee24-4b84-af01-6ff32fddde46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:10.469957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2dcaf24f-ee24-4b84-af01-6ff32fddde46 None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:10.471872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2dcaf24f-ee24-4b84-af01-6ff32fddde46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:10.521640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:10.596716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 678/1359] 10.222.0.55 () {58 vars in 1058 bytes} [Tue Sep 27 17:17:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:17:10.606028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7bbdc-3e0c-4290-a70b-475abdb94131 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:10.607568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f2c7bbdc-3e0c-4290-a70b-475abdb94131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:10.771774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7bbdc-3e0c-4290-a70b-475abdb94131 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:10.849625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:10.892311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc8756-afea-4955-b445-a4e12bd8bdb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:10.892964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc8756-afea-4955-b445-a4e12bd8bdb4 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:10.893242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc8756-afea-4955-b445-a4e12bd8bdb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:10.893475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc8756-afea-4955-b445-a4e12bd8bdb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:10.895075 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2cdc8756-afea-4955-b445-a4e12bd8bdb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:11.000262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc8756-afea-4955-b445-a4e12bd8bdb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:11.003207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc8756-afea-4955-b445-a4e12bd8bdb4 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:11.024722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 682/1360] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:13.418628 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a454514-7c2b-4e63-9a5e-7594fd03fb65 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:13.419163 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a454514-7c2b-4e63-9a5e-7594fd03fb65 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:13.419163 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a454514-7c2b-4e63-9a5e-7594fd03fb65 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:13.420591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 679/1361] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:13.426447 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06a120-c574-424c-a1bc-0a019b330d37 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:13.426900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06a120-c574-424c-a1bc-0a019b330d37 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:13.427075 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06a120-c574-424c-a1bc-0a019b330d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:13.495537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1a06a120-c574-424c-a1bc-0a019b330d37 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:13.497340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1a06a120-c574-424c-a1bc-0a019b330d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:13.539088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:13.603924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 683/1362] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:13.897529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a454514-7c2b-4e63-9a5e-7594fd03fb65 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:13.899096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0a454514-7c2b-4e63-9a5e-7594fd03fb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:14.074182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a454514-7c2b-4e63-9a5e-7594fd03fb65 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:14.086444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:14.224137 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658739e8-d207-442f-b513-7adc8c10d55c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:14.225036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658739e8-d207-442f-b513-7adc8c10d55c service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:14.225440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658739e8-d207-442f-b513-7adc8c10d55c service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:14.225834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658739e8-d207-442f-b513-7adc8c10d55c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:14.228913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-658739e8-d207-442f-b513-7adc8c10d55c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:14.324363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658739e8-d207-442f-b513-7adc8c10d55c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:14.325963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658739e8-d207-442f-b513-7adc8c10d55c service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:14.345010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 680/1363] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:16.864352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238ad4b-6f13-4e7a-a65d-acc1c0231953 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:16.865225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238ad4b-6f13-4e7a-a65d-acc1c0231953 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:16.865501 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238ad4b-6f13-4e7a-a65d-acc1c0231953 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:16.867120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 684/1364] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:16.872232 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750f17a-611f-4062-970d-55f4d973065d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:16.872664 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750f17a-611f-4062-970d-55f4d973065d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:16.872855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750f17a-611f-4062-970d-55f4d973065d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:16.888638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:16.944900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0750f17a-611f-4062-970d-55f4d973065d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:16.946322 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0750f17a-611f-4062-970d-55f4d973065d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:17.051004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 681/1365] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:17:17.342638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405f7ad-e7b8-4fbf-9448-9e1f6903e792 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:17.342980 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405f7ad-e7b8-4fbf-9448-9e1f6903e792 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:17.343233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405f7ad-e7b8-4fbf-9448-9e1f6903e792 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:17.344908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 685/1366] 10.222.0.55 () {54 vars in 961 bytes} [Tue Sep 27 17:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:17.351433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f0271-2580-477e-a4d5-bddde6f060f3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:17.351932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f0271-2580-477e-a4d5-bddde6f060f3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:17.352205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f0271-2580-477e-a4d5-bddde6f060f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:17.434297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-4d2f0271-2580-477e-a4d5-bddde6f060f3 None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:17.435495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4d2f0271-2580-477e-a4d5-bddde6f060f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:17.472430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:17.559148 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 682/1367] 10.222.0.55 () {58 vars in 1058 bytes} [Tue Sep 27 17:17:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:17.568019 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405f7ad-e7b8-4fbf-9448-9e1f6903e792 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:17.569641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1405f7ad-e7b8-4fbf-9448-9e1f6903e792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:17.730929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405f7ad-e7b8-4fbf-9448-9e1f6903e792 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:17.752811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:17.858660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658dd27c-91e1-404b-aade-d0ed33085d5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:17.859265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd27c-91e1-404b-aade-d0ed33085d5d service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:17.859536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd27c-91e1-404b-aade-d0ed33085d5d service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:17.859803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658dd27c-91e1-404b-aade-d0ed33085d5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:17.862213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-658dd27c-91e1-404b-aade-d0ed33085d5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:17.973075 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658dd27c-91e1-404b-aade-d0ed33085d5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:17.975681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658dd27c-91e1-404b-aade-d0ed33085d5d service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:17.996774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 686/1368] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:18.055520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e91af8-de4c-46aa-8ee0-6cf9376366c2 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:18.056016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e91af8-de4c-46aa-8ee0-6cf9376366c2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:18.056188 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e91af8-de4c-46aa-8ee0-6cf9376366c2 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:18.057466 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 683/1369] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:18.063110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45872515-fc0b-4202-b826-012718c2e524 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:18.063398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45872515-fc0b-4202-b826-012718c2e524 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:18.064262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45872515-fc0b-4202-b826-012718c2e524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:18.097956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:18.153399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-45872515-fc0b-4202-b826-012718c2e524 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:18.154803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-45872515-fc0b-4202-b826-012718c2e524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:18.274109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 687/1370] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:18.629740 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8a1a9-1d0f-4dc0-a6ff-1878bd40bec4 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:18.630021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8a1a9-1d0f-4dc0-a6ff-1878bd40bec4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:18.630412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8a1a9-1d0f-4dc0-a6ff-1878bd40bec4 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:18.631895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 684/1371] 10.222.0.55 () {54 vars in 963 bytes} [Tue Sep 27 17:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:18.636638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1003e-4dbd-4356-9f18-27c8ea16747d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:18.636921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1003e-4dbd-4356-9f18-27c8ea16747d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:18.637191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1003e-4dbd-4356-9f18-27c8ea16747d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:18.725337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-faa1003e-4dbd-4356-9f18-27c8ea16747d None None] MFA Rules not processed for user `528981e41f0b436b82989d62c713d993`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:18.726066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-faa1003e-4dbd-4356-9f18-27c8ea16747d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:18.786487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:18.817871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 688/1372] 10.222.0.55 () {58 vars in 1060 bytes} [Tue Sep 27 17:17:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:18.825949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8a1a9-1d0f-4dc0-a6ff-1878bd40bec4 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:18.827605 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-73a8a1a9-1d0f-4dc0-a6ff-1878bd40bec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:18.936936 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8a1a9-1d0f-4dc0-a6ff-1878bd40bec4 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:18.983620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:19.026226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2c96f-63ae-47c0-b0df-e912c37eb96c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:19.026848 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2c96f-63ae-47c0-b0df-e912c37eb96c service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:19.027125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2c96f-63ae-47c0-b0df-e912c37eb96c service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:19.027391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2c96f-63ae-47c0-b0df-e912c37eb96c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:19.029031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a5e2c96f-63ae-47c0-b0df-e912c37eb96c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:19.171520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2c96f-63ae-47c0-b0df-e912c37eb96c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:19.173211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2c96f-63ae-47c0-b0df-e912c37eb96c service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:19.195824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 685/1373] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:20.270591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2092889-ad6c-46b0-b129-52190695563d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:20.271148 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2092889-ad6c-46b0-b129-52190695563d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:20.271244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2092889-ad6c-46b0-b129-52190695563d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:20.272519 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 689/1374] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:20.277782 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590cdbd-079d-44ea-a1a7-2d89e3e1c92a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:20.278225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590cdbd-079d-44ea-a1a7-2d89e3e1c92a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:20.278407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590cdbd-079d-44ea-a1a7-2d89e3e1c92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:20.344931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5590cdbd-079d-44ea-a1a7-2d89e3e1c92a None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:20.346172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5590cdbd-079d-44ea-a1a7-2d89e3e1c92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:20.367388 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:20.445593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 686/1375] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:20.743435 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2092889-ad6c-46b0-b129-52190695563d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:20.744983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e2092889-ad6c-46b0-b129-52190695563d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:20.903534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2092889-ad6c-46b0-b129-52190695563d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:20.957563 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:21.021189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6676aee-1f23-4eec-91f1-6b144ba85ba3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:21.021697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6676aee-1f23-4eec-91f1-6b144ba85ba3 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:21.021972 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6676aee-1f23-4eec-91f1-6b144ba85ba3 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:21.022276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6676aee-1f23-4eec-91f1-6b144ba85ba3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:21.023859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a6676aee-1f23-4eec-91f1-6b144ba85ba3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:21.118805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6676aee-1f23-4eec-91f1-6b144ba85ba3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:21.120493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6676aee-1f23-4eec-91f1-6b144ba85ba3 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:21.139955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 690/1376] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:23.359184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9673317-9f5e-4ffb-90f1-561e14df9a9b None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:23.359745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9673317-9f5e-4ffb-90f1-561e14df9a9b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:23.359745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9673317-9f5e-4ffb-90f1-561e14df9a9b None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:23.361749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 687/1377] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:23.366856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85884e-1d60-4d80-9dd9-d42cf3a127f5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:23.367256 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85884e-1d60-4d80-9dd9-d42cf3a127f5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:23.367432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85884e-1d60-4d80-9dd9-d42cf3a127f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:23.439602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fb85884e-1d60-4d80-9dd9-d42cf3a127f5 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:23.440948 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fb85884e-1d60-4d80-9dd9-d42cf3a127f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:23.448711 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:23.558121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 691/1378] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:23.856761 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9673317-9f5e-4ffb-90f1-561e14df9a9b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:23.858265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b9673317-9f5e-4ffb-90f1-561e14df9a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:24.010622 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9673317-9f5e-4ffb-90f1-561e14df9a9b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:24.014814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:24.135356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7685992-f26a-4f7a-be72-6c07db41bf4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:24.136030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7685992-f26a-4f7a-be72-6c07db41bf4c service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:24.136363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7685992-f26a-4f7a-be72-6c07db41bf4c service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:24.136692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7685992-f26a-4f7a-be72-6c07db41bf4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:24.138341 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d7685992-f26a-4f7a-be72-6c07db41bf4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:24.232549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7685992-f26a-4f7a-be72-6c07db41bf4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:24.234200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7685992-f26a-4f7a-be72-6c07db41bf4c service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:24.253264 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 688/1379] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:26.498382 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780fb12-d7c1-477d-87f7-fb385f119e96 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:26.498859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780fb12-d7c1-477d-87f7-fb385f119e96 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:26.499058 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780fb12-d7c1-477d-87f7-fb385f119e96 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:26.500542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 692/1380] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:26.506002 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a647be-200e-4558-bd4d-ba31920412d1 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:26.506418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a647be-200e-4558-bd4d-ba31920412d1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:26.506572 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a647be-200e-4558-bd4d-ba31920412d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:26.519091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:26.581941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a4a647be-200e-4558-bd4d-ba31920412d1 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:26.583123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a4a647be-200e-4558-bd4d-ba31920412d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:26.698854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 689/1381] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:26.980321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780fb12-d7c1-477d-87f7-fb385f119e96 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:26.981887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e780fb12-d7c1-477d-87f7-fb385f119e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:27.123289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:27.134756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780fb12-d7c1-477d-87f7-fb385f119e96 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:27.258809 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce589a5-d805-4157-9bcd-c20cdb2aa6d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:27.259334 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce589a5-d805-4157-9bcd-c20cdb2aa6d3 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:27.259637 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce589a5-d805-4157-9bcd-c20cdb2aa6d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:27.259905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce589a5-d805-4157-9bcd-c20cdb2aa6d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:27.261529 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5ce589a5-d805-4157-9bcd-c20cdb2aa6d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:27.364447 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce589a5-d805-4157-9bcd-c20cdb2aa6d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:27.366059 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce589a5-d805-4157-9bcd-c20cdb2aa6d3 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:27.376233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:27.386450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 693/1382] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:29.574395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718af238-2e40-4f59-a445-cfc01a9c352a None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:29.574913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718af238-2e40-4f59-a445-cfc01a9c352a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:29.574913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718af238-2e40-4f59-a445-cfc01a9c352a None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:29.576292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 690/1383] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:29.582681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9ba86-d03c-49fa-9187-9dcb727fce06 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:29.583100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9ba86-d03c-49fa-9187-9dcb727fce06 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:29.583387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9ba86-d03c-49fa-9187-9dcb727fce06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:29.666081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-52e9ba86-d03c-49fa-9187-9dcb727fce06 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:29.667708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-52e9ba86-d03c-49fa-9187-9dcb727fce06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:29.776207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 694/1384] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:30.066929 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718af238-2e40-4f59-a445-cfc01a9c352a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:30.068528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-718af238-2e40-4f59-a445-cfc01a9c352a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:30.159973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:30.220933 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718af238-2e40-4f59-a445-cfc01a9c352a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:30.339235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339b2a8-360b-461e-a93c-f7679c42e10e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:30.339840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339b2a8-360b-461e-a93c-f7679c42e10e service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:30.340115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339b2a8-360b-461e-a93c-f7679c42e10e service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:30.340383 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339b2a8-360b-461e-a93c-f7679c42e10e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:30.341984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4339b2a8-360b-461e-a93c-f7679c42e10e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:30.392850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:30.432618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339b2a8-360b-461e-a93c-f7679c42e10e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:30.434166 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339b2a8-360b-461e-a93c-f7679c42e10e service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:30.451738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 691/1385] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:32.695527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803d17b-ebf2-4383-b60a-b5a5a2ba56fb None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:32.696136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803d17b-ebf2-4383-b60a-b5a5a2ba56fb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:32.696136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803d17b-ebf2-4383-b60a-b5a5a2ba56fb None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:32.697403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 695/1386] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:32.702701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6630b-e3cc-404f-bb48-e066b5c0ab6c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:32.703162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6630b-e3cc-404f-bb48-e066b5c0ab6c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:32.703283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6630b-e3cc-404f-bb48-e066b5c0ab6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:32.768971 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-84f6630b-e3cc-404f-bb48-e066b5c0ab6c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:32.770274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-84f6630b-e3cc-404f-bb48-e066b5c0ab6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:32.869319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 692/1387] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:33.157655 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803d17b-ebf2-4383-b60a-b5a5a2ba56fb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:33.159219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1803d17b-ebf2-4383-b60a-b5a5a2ba56fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:33.241085 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:33.311738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803d17b-ebf2-4383-b60a-b5a5a2ba56fb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:33.427271 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1843ccf-4ad6-436f-a050-dd4bb344ddd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:33.427784 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1843ccf-4ad6-436f-a050-dd4bb344ddd5 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:33.428055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1843ccf-4ad6-436f-a050-dd4bb344ddd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:33.428321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1843ccf-4ad6-436f-a050-dd4bb344ddd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:33.429893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c1843ccf-4ad6-436f-a050-dd4bb344ddd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:33.467419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:33.519776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1843ccf-4ad6-436f-a050-dd4bb344ddd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:33.521404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1843ccf-4ad6-436f-a050-dd4bb344ddd5 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:33.543503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 696/1388] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:34.571858 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644604f-6b60-490e-b2f2-d4a3d1c20364 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:34.572378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644604f-6b60-490e-b2f2-d4a3d1c20364 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:34.572651 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644604f-6b60-490e-b2f2-d4a3d1c20364 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:34.574211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 693/1389] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:34.579448 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffde4cb-9e42-4c64-a614-1b2e4a24b201 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:34.579923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffde4cb-9e42-4c64-a614-1b2e4a24b201 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:34.580017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffde4cb-9e42-4c64-a614-1b2e4a24b201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:34.650995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bffde4cb-9e42-4c64-a614-1b2e4a24b201 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:34.652186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bffde4cb-9e42-4c64-a614-1b2e4a24b201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:34.750022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 697/1390] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:35.056322 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf03dc-e936-4f28-824a-0202ae1ea728 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:35.056776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf03dc-e936-4f28-824a-0202ae1ea728 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:35.057247 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf03dc-e936-4f28-824a-0202ae1ea728 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:35.058865 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 694/1391] 10.222.0.55 () {54 vars in 962 bytes} [Tue Sep 27 17:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:35.065395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8ff65-af29-48b1-8a52-91fc7b4b3254 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:35.065657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8ff65-af29-48b1-8a52-91fc7b4b3254 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:35.065925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c8ff65-af29-48b1-8a52-91fc7b4b3254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:35.081486 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:35.142746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e3c8ff65-af29-48b1-8a52-91fc7b4b3254 None None] MFA Rules not processed for user `6cc89c1a7e684612a73ec9fef6bda780`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:35.144233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e3c8ff65-af29-48b1-8a52-91fc7b4b3254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:35.220367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 698/1392] 10.222.0.55 () {58 vars in 1059 bytes} [Tue Sep 27 17:17:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:35.229448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf03dc-e936-4f28-824a-0202ae1ea728 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:35.230985 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9cdf03dc-e936-4f28-824a-0202ae1ea728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:35.262593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:35.340919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf03dc-e936-4f28-824a-0202ae1ea728 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:35.414426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f5da1-2949-4124-87db-5a370807f7ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc89c1a7e684612a73ec9fef6bda780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:35.415032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f5da1-2949-4124-87db-5a370807f7ec service glance] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:35.415308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f5da1-2949-4124-87db-5a370807f7ec service glance] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:35.415574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f5da1-2949-4124-87db-5a370807f7ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:35.417157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ab7f5da1-2949-4124-87db-5a370807f7ec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:35.499721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:35.532547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f5da1-2949-4124-87db-5a370807f7ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:35.533980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f5da1-2949-4124-87db-5a370807f7ec service glance] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:35.552193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 695/1393] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:35.867516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b422dfd-5f67-4303-b930-da1ffcfba745 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:35.868095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b422dfd-5f67-4303-b930-da1ffcfba745 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:35.868095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b422dfd-5f67-4303-b930-da1ffcfba745 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:35.869418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 699/1394] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:35.875857 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd17a9e-eef0-4b7d-af35-b507d5463356 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:35.876410 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd17a9e-eef0-4b7d-af35-b507d5463356 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:35.876657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd17a9e-eef0-4b7d-af35-b507d5463356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:35.949340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-cbd17a9e-eef0-4b7d-af35-b507d5463356 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:35.950921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cbd17a9e-eef0-4b7d-af35-b507d5463356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:36.051617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 696/1395] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:36.288245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa5b30-928d-4f98-a159-5a57c945afb9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:36.288531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa5b30-928d-4f98-a159-5a57c945afb9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:36.288851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa5b30-928d-4f98-a159-5a57c945afb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:36.346652 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd17a9e-eef0-4b7d-af35-b507d5463356 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:36.348010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cbd17a9e-eef0-4b7d-af35-b507d5463356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:36.358667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-30aa5b30-928d-4f98-a159-5a57c945afb9 None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:36.359923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-30aa5b30-928d-4f98-a159-5a57c945afb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:36.388352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:36.415306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:36.447550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 700/1396] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:17:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:36.465716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd17a9e-eef0-4b7d-af35-b507d5463356 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:36.492537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74723bfd-a02b-4551-b7a4-54f58d766c77 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:36.492820 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74723bfd-a02b-4551-b7a4-54f58d766c77 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:36.493089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74723bfd-a02b-4551-b7a4-54f58d766c77 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:36.494486 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 701/1397] 10.222.0.55 () {54 vars in 943 bytes} [Tue Sep 27 17:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:17:36.500329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f30f2-40e5-4f9d-bcc9-237adf87d24d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:36.501523 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f30f2-40e5-4f9d-bcc9-237adf87d24d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:36.501799 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f30f2-40e5-4f9d-bcc9-237adf87d24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:36.580556 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5c2f30f2-40e5-4f9d-bcc9-237adf87d24d None None] MFA Rules not processed for user `39073006a0054b56bb1bba46d2d0768e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:36.581762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5c2f30f2-40e5-4f9d-bcc9-237adf87d24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:36.584272 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa075b77-9db4-41ce-bcfd-05efb5a13706 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:36.584872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa075b77-9db4-41ce-bcfd-05efb5a13706 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:36.585147 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa075b77-9db4-41ce-bcfd-05efb5a13706 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:36.585414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa075b77-9db4-41ce-bcfd-05efb5a13706 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:36.587004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fa075b77-9db4-41ce-bcfd-05efb5a13706 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:36.621602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:36.679112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa075b77-9db4-41ce-bcfd-05efb5a13706 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:36.680719 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa075b77-9db4-41ce-bcfd-05efb5a13706 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:36.691736 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 702/1398] 10.222.0.55 () {58 vars in 1040 bytes} [Tue Sep 27 17:17:36 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:17:36.699494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 697/1399] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:17:36.700987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f30f2-40e5-4f9d-bcc9-237adf87d24d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:36.710698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f30f2-40e5-4f9d-bcc9-237adf87d24d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:36.739248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:36.827527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be6960-ab33-42df-b240-80a76fa0ad15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:36.828134 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be6960-ab33-42df-b240-80a76fa0ad15 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:36.828410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be6960-ab33-42df-b240-80a76fa0ad15 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:36.828707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be6960-ab33-42df-b240-80a76fa0ad15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:36.830505 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-60be6960-ab33-42df-b240-80a76fa0ad15 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:36.925435 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be6960-ab33-42df-b240-80a76fa0ad15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:36.927084 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be6960-ab33-42df-b240-80a76fa0ad15 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:36.935919 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 703/1400] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:17:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 17:17:38.925259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee932b-901d-4f92-b889-68abc520bc7b None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:38.925259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee932b-901d-4f92-b889-68abc520bc7b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:38.925259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee932b-901d-4f92-b889-68abc520bc7b None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:38.929998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 698/1401] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:38.933867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1f278-f415-4d9e-bb08-9943043fcdc6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:38.934366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1f278-f415-4d9e-bb08-9943043fcdc6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:38.934468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1f278-f415-4d9e-bb08-9943043fcdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:39.007357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8dd1f278-f415-4d9e-bb08-9943043fcdc6 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:39.008898 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8dd1f278-f415-4d9e-bb08-9943043fcdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:39.016959 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:39.120946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 704/1402] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:39.423742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee932b-901d-4f92-b889-68abc520bc7b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:39.425289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-16ee932b-901d-4f92-b889-68abc520bc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:39.596687 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee932b-901d-4f92-b889-68abc520bc7b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:39.611300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:39.733615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e66d3d-0e94-4361-819b-a4e8ccc85bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:39.734251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e66d3d-0e94-4361-819b-a4e8ccc85bcd service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:39.734534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e66d3d-0e94-4361-819b-a4e8ccc85bcd service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:39.734831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e66d3d-0e94-4361-819b-a4e8ccc85bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:39.736551 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b7e66d3d-0e94-4361-819b-a4e8ccc85bcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:39.843949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e66d3d-0e94-4361-819b-a4e8ccc85bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:39.846167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e66d3d-0e94-4361-819b-a4e8ccc85bcd service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:39.865728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 699/1403] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:42.113766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed3920-95f8-4f09-a034-d75c3e3ed297 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:42.114462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed3920-95f8-4f09-a034-d75c3e3ed297 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:42.114462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed3920-95f8-4f09-a034-d75c3e3ed297 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:42.122248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 705/1404] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:42.126116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac76cd-c9aa-4409-befa-4d942fa4a01d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:42.126590 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac76cd-c9aa-4409-befa-4d942fa4a01d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:42.126673 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac76cd-c9aa-4409-befa-4d942fa4a01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:42.174813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:42.246207 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-aeac76cd-c9aa-4409-befa-4d942fa4a01d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:42.247657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-aeac76cd-c9aa-4409-befa-4d942fa4a01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:42.399185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 700/1405] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:42.692411 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed3920-95f8-4f09-a034-d75c3e3ed297 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:42.693983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-35ed3920-95f8-4f09-a034-d75c3e3ed297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:42.850065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:42.862473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed3920-95f8-4f09-a034-d75c3e3ed297 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:42.992580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857f9cb-a2ad-4629-a32d-6fe16d6ec5e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:42.993120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f9cb-a2ad-4629-a32d-6fe16d6ec5e7 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:42.993387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f9cb-a2ad-4629-a32d-6fe16d6ec5e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:42.993646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857f9cb-a2ad-4629-a32d-6fe16d6ec5e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:42.995277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3857f9cb-a2ad-4629-a32d-6fe16d6ec5e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:43.098566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857f9cb-a2ad-4629-a32d-6fe16d6ec5e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:43.100170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857f9cb-a2ad-4629-a32d-6fe16d6ec5e7 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:43.110910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:43.119995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 706/1406] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:45.329118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafa802-e86c-4897-8100-407208dd6489 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:45.329647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafa802-e86c-4897-8100-407208dd6489 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:45.329647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafa802-e86c-4897-8100-407208dd6489 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:45.331168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 701/1407] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:45.336174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf047a-92a9-4db8-a9f1-9ef5ae326e8d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:45.336583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf047a-92a9-4db8-a9f1-9ef5ae326e8d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:45.336739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf047a-92a9-4db8-a9f1-9ef5ae326e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:45.405036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a9cf047a-92a9-4db8-a9f1-9ef5ae326e8d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:45.406274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a9cf047a-92a9-4db8-a9f1-9ef5ae326e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:45.507151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 707/1408] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:45.792851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafa802-e86c-4897-8100-407208dd6489 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:17:45.794373 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4cafa802-e86c-4897-8100-407208dd6489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:45.897893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:45.959125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafa802-e86c-4897-8100-407208dd6489 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:17:46.086169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5cf46-b1c5-4ab7-ada0-ec147a809396 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:17:46.087350 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5cf46-b1c5-4ab7-ada0-ec147a809396 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:46.087632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5cf46-b1c5-4ab7-ada0-ec147a809396 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:46.087919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5cf46-b1c5-4ab7-ada0-ec147a809396 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:46.089547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-66d5cf46-b1c5-4ab7-ada0-ec147a809396 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:46.153760 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:46.189825 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5cf46-b1c5-4ab7-ada0-ec147a809396 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:17:46.191638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5cf46-b1c5-4ab7-ada0-ec147a809396 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:17:46.211846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 702/1409] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:17:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:17:55.505292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=67 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:56.209193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=20 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:59.633233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587b6a9-9772-4bdd-a51a-e0efecaff4de None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:59.634253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587b6a9-9772-4bdd-a51a-e0efecaff4de None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:59.634541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587b6a9-9772-4bdd-a51a-e0efecaff4de None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:59.635985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 708/1410] 10.222.0.55 () {54 vars in 918 bytes} [Tue Sep 27 17:17:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:59.641305 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cee51-5e39-40ed-85ea-cc1ab4c418a3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:59.641717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cee51-5e39-40ed-85ea-cc1ab4c418a3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:59.641846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cee51-5e39-40ed-85ea-cc1ab4c418a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:59.717225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c20cee51-5e39-40ed-85ea-cc1ab4c418a3 None None] MFA Rules not processed for user `3b16c430b870424c8d366c9a2959003e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:59.718987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c20cee51-5e39-40ed-85ea-cc1ab4c418a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:17:59.823230 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 703/1411] 10.222.0.55 () {58 vars in 1015 bytes} [Tue Sep 27 17:17:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:17:59.837707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e58376-2775-4632-864b-75cf68925d81 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:59.837989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e58376-2775-4632-864b-75cf68925d81 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:59.838285 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e58376-2775-4632-864b-75cf68925d81 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:59.839642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 709/1412] 10.222.0.55 () {54 vars in 918 bytes} [Tue Sep 27 17:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:17:59.844857 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f06bb9-6990-42d9-800d-77b8cd3dc6f7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:17:59.845139 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f06bb9-6990-42d9-800d-77b8cd3dc6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:17:59.845411 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f06bb9-6990-42d9-800d-77b8cd3dc6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:17:59.917198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:17:59.919678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e6f06bb9-6990-42d9-800d-77b8cd3dc6f7 None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:17:59.921464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e6f06bb9-6990-42d9-800d-77b8cd3dc6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:00.036823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 704/1413] 10.222.0.55 () {58 vars in 1015 bytes} [Tue Sep 27 17:17:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:02.642689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631a3de-f38a-413a-bec3-fb4400963856 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:02.643243 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631a3de-f38a-413a-bec3-fb4400963856 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:02.643243 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631a3de-f38a-413a-bec3-fb4400963856 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:02.644601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:18:02.649953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b293e05-cb54-489c-a999-4f811d590bf7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:02.650411 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b293e05-cb54-489c-a999-4f811d590bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:02.650507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b293e05-cb54-489c-a999-4f811d590bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:02.714569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3b293e05-cb54-489c-a999-4f811d590bf7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:02.715757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3b293e05-cb54-489c-a999-4f811d590bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:02.769493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:02.809536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:03.107022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631a3de-f38a-413a-bec3-fb4400963856 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:03.108559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b631a3de-f38a-413a-bec3-fb4400963856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:03.252051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631a3de-f38a-413a-bec3-fb4400963856 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:03.323645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:03.365083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ce68d-2673-4f78-9511-d1b532b54d3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:03.365673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ce68d-2673-4f78-9511-d1b532b54d3a service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:03.365942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ce68d-2673-4f78-9511-d1b532b54d3a service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:03.366240 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ce68d-2673-4f78-9511-d1b532b54d3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:03.367818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d69ce68d-2673-4f78-9511-d1b532b54d3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:03.454978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ce68d-2673-4f78-9511-d1b532b54d3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:03.456542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ce68d-2673-4f78-9511-d1b532b54d3a service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:03.473780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 711/1416] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:04.617152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbf32d-aeda-4d56-beb1-280c23d7ee8d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:04.617697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbf32d-aeda-4d56-beb1-280c23d7ee8d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:04.617796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbbf32d-aeda-4d56-beb1-280c23d7ee8d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:04.619151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 17:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:04.625762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823827e1-d32c-4426-8a27-1be95fb0289f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:04.626259 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823827e1-d32c-4426-8a27-1be95fb0289f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:04.626375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823827e1-d32c-4426-8a27-1be95fb0289f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:04.700885 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-823827e1-d32c-4426-8a27-1be95fb0289f None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:04.702099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-823827e1-d32c-4426-8a27-1be95fb0289f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:04.730356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:04.798977 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 17:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:04.951320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbf32d-aeda-4d56-beb1-280c23d7ee8d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:04.952782 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-bcbbf32d-aeda-4d56-beb1-280c23d7ee8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:05.098277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbbf32d-aeda-4d56-beb1-280c23d7ee8d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:05.143482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:05.221423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a83f1-aff5-4607-884b-168fc45f2b7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:05.221940 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a83f1-aff5-4607-884b-168fc45f2b7e service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:05.222243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a83f1-aff5-4607-884b-168fc45f2b7e service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:05.222509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a83f1-aff5-4607-884b-168fc45f2b7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:05.224106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0c0a83f1-aff5-4607-884b-168fc45f2b7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:05.428049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a83f1-aff5-4607-884b-168fc45f2b7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:05.429640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a83f1-aff5-4607-884b-168fc45f2b7e service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:05.447781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 707/1419] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:06.584216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795ab47-93ca-4328-a016-ee0dafe373d7 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:06.584737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795ab47-93ca-4328-a016-ee0dafe373d7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:06.584737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795ab47-93ca-4328-a016-ee0dafe373d7 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:06.586021 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 17:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:06.592118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be286c-32b7-4b84-b352-226262a48bb1 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:06.592537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be286c-32b7-4b84-b352-226262a48bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:06.592660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be286c-32b7-4b84-b352-226262a48bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:06.646802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:06.668711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e1be286c-32b7-4b84-b352-226262a48bb1 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:06.669911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e1be286c-32b7-4b84-b352-226262a48bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:06.765708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 17:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:06.924247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795ab47-93ca-4328-a016-ee0dafe373d7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:06.925759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9795ab47-93ca-4328-a016-ee0dafe373d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:07.070906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795ab47-93ca-4328-a016-ee0dafe373d7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:07.083083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:07.183555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe481e-7288-4a59-b656-b0b3ea1290c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:07.184067 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe481e-7288-4a59-b656-b0b3ea1290c4 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:07.184336 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe481e-7288-4a59-b656-b0b3ea1290c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:07.184598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe481e-7288-4a59-b656-b0b3ea1290c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:07.186189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0bbe481e-7288-4a59-b656-b0b3ea1290c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:07.276310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe481e-7288-4a59-b656-b0b3ea1290c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:07.278206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe481e-7288-4a59-b656-b0b3ea1290c4 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:07.297044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 714/1422] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:08.398295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb691e-888a-4271-9d59-a83398ae94d2 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:08.398806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb691e-888a-4271-9d59-a83398ae94d2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:08.398915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb691e-888a-4271-9d59-a83398ae94d2 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:08.400185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 17:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:08.407029 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b006704-365c-48d6-a6e0-1535ff205c34 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:08.407458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b006704-365c-48d6-a6e0-1535ff205c34 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:08.407553 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b006704-365c-48d6-a6e0-1535ff205c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:08.427628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:08.482639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4b006704-365c-48d6-a6e0-1535ff205c34 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:08.483847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4b006704-365c-48d6-a6e0-1535ff205c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:08.577375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 17:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:08.729460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb691e-888a-4271-9d59-a83398ae94d2 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:08.730930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-16cb691e-888a-4271-9d59-a83398ae94d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:08.838853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:08.875821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb691e-888a-4271-9d59-a83398ae94d2 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:08.988041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25027eb7-1ff5-43c2-bd13-f04305b904b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:08.988582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25027eb7-1ff5-43c2-bd13-f04305b904b5 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:08.988855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25027eb7-1ff5-43c2-bd13-f04305b904b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:08.989121 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25027eb7-1ff5-43c2-bd13-f04305b904b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:08.990729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-25027eb7-1ff5-43c2-bd13-f04305b904b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:09.068153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:09.085319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25027eb7-1ff5-43c2-bd13-f04305b904b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:09.086953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25027eb7-1ff5-43c2-bd13-f04305b904b5 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:09.105475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 710/1425] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:10.231904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aba1ef-3789-490e-8e8f-f4f206965f12 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:10.232478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aba1ef-3789-490e-8e8f-f4f206965f12 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:10.232730 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aba1ef-3789-490e-8e8f-f4f206965f12 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:10.235373 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 17:18:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:10.241913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29883c1-a043-4d81-9311-da4f64579520 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:10.242424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29883c1-a043-4d81-9311-da4f64579520 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:10.242521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29883c1-a043-4d81-9311-da4f64579520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:10.322300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e29883c1-a043-4d81-9311-da4f64579520 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:10.323512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e29883c1-a043-4d81-9311-da4f64579520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:10.418558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 17:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:10.569703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aba1ef-3789-490e-8e8f-f4f206965f12 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:10.571322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-47aba1ef-3789-490e-8e8f-f4f206965f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:10.655335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:10.717970 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aba1ef-3789-490e-8e8f-f4f206965f12 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:10.832200 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c472912-9a44-4c53-af4c-d8ff73e304d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:10.832784 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c472912-9a44-4c53-af4c-d8ff73e304d2 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:10.833057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c472912-9a44-4c53-af4c-d8ff73e304d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:10.833322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c472912-9a44-4c53-af4c-d8ff73e304d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:10.834975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4c472912-9a44-4c53-af4c-d8ff73e304d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:10.883703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:10.925182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c472912-9a44-4c53-af4c-d8ff73e304d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:10.926818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c472912-9a44-4c53-af4c-d8ff73e304d2 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:10.944981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 717/1428] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:12.011879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903fae8-03d9-4521-8969-fd206ed21d37 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:12.012418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903fae8-03d9-4521-8969-fd206ed21d37 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:12.012418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903fae8-03d9-4521-8969-fd206ed21d37 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:12.013683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 17:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:12.020405 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bf6cf-cf2a-4dba-ba06-e82ea83e8353 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:12.020907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bf6cf-cf2a-4dba-ba06-e82ea83e8353 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:12.021002 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bf6cf-cf2a-4dba-ba06-e82ea83e8353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:12.098485 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f53bf6cf-cf2a-4dba-ba06-e82ea83e8353 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:12.099698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f53bf6cf-cf2a-4dba-ba06-e82ea83e8353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:12.194679 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 17:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:12.351615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903fae8-03d9-4521-8969-fd206ed21d37 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:12.353371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e903fae8-03d9-4521-8969-fd206ed21d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:12.401365 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:12.500100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903fae8-03d9-4521-8969-fd206ed21d37 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:12.618807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b4f25-918e-430a-9d66-9aeae8f6b088 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:12.619411 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b4f25-918e-430a-9d66-9aeae8f6b088 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:12.619685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b4f25-918e-430a-9d66-9aeae8f6b088 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:12.619972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b4f25-918e-430a-9d66-9aeae8f6b088 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:12.621563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-181b4f25-918e-430a-9d66-9aeae8f6b088 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:12.629089 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:12.711418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b4f25-918e-430a-9d66-9aeae8f6b088 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:12.713014 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b4f25-918e-430a-9d66-9aeae8f6b088 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:12.730866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 713/1431] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:13.778221 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e3640-0aff-4149-bef7-be7a67721353 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:13.778769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e3640-0aff-4149-bef7-be7a67721353 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:13.778866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e3640-0aff-4149-bef7-be7a67721353 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:13.780199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 17:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:13.786544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b6639-da3d-4ce3-9927-eac669e92c49 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:13.786960 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b6639-da3d-4ce3-9927-eac669e92c49 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:13.787117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b6639-da3d-4ce3-9927-eac669e92c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:13.862567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-362b6639-da3d-4ce3-9927-eac669e92c49 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:13.863760 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-362b6639-da3d-4ce3-9927-eac669e92c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:13.929684 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:13.959370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 17:18:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:14.110397 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e3640-0aff-4149-bef7-be7a67721353 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:14.111916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-370e3640-0aff-4149-bef7-be7a67721353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:14.129711 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:14.255981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e3640-0aff-4149-bef7-be7a67721353 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:14.341976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:14.368712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed27d00-29f2-4f27-ab09-0ec55e7a951c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:14.369248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed27d00-29f2-4f27-ab09-0ec55e7a951c service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:14.369524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed27d00-29f2-4f27-ab09-0ec55e7a951c service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:14.369795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed27d00-29f2-4f27-ab09-0ec55e7a951c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:14.371881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6ed27d00-29f2-4f27-ab09-0ec55e7a951c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:14.469187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed27d00-29f2-4f27-ab09-0ec55e7a951c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:14.470815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed27d00-29f2-4f27-ab09-0ec55e7a951c service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:14.489971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 720/1434] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:15.545426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e7f71-ea84-40a1-be03-1afa4397b621 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:15.545977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e7f71-ea84-40a1-be03-1afa4397b621 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:15.545977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e7f71-ea84-40a1-be03-1afa4397b621 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:15.547416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 17:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:15.553530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d6bd1-702c-4378-938e-c72b02e280ec None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:15.553928 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d6bd1-702c-4378-938e-c72b02e280ec None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:15.554091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d6bd1-702c-4378-938e-c72b02e280ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:15.628422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c60d6bd1-702c-4378-938e-c72b02e280ec None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:15.629615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c60d6bd1-702c-4378-938e-c72b02e280ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:15.670346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:15.725274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 17:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:15.876665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e7f71-ea84-40a1-be03-1afa4397b621 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:15.878276 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ab4e7f71-ea84-40a1-be03-1afa4397b621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:16.023305 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e7f71-ea84-40a1-be03-1afa4397b621 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:16.075539 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:16.133097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6d922-9874-46a7-8d11-5279ba2b2533 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:16.133686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6d922-9874-46a7-8d11-5279ba2b2533 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:16.133955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6d922-9874-46a7-8d11-5279ba2b2533 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:16.134244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6d922-9874-46a7-8d11-5279ba2b2533 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:16.135804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-59d6d922-9874-46a7-8d11-5279ba2b2533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:16.223814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6d922-9874-46a7-8d11-5279ba2b2533 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:16.225390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6d922-9874-46a7-8d11-5279ba2b2533 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:16.242844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 716/1437] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:17.293128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2357b0-6653-49ea-889e-63610f414ad8 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:17.293718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2357b0-6653-49ea-889e-63610f414ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:17.293718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2357b0-6653-49ea-889e-63610f414ad8 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:17.295068 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 17:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:17.301763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6d334-db3e-4f3a-9702-cd6f4463d27b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:17.302204 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6d334-db3e-4f3a-9702-cd6f4463d27b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:17.302369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6d334-db3e-4f3a-9702-cd6f4463d27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:17.377700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-cce6d334-db3e-4f3a-9702-cd6f4463d27b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:17.378942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cce6d334-db3e-4f3a-9702-cd6f4463d27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:17.380907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:17.473659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 17:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:17.627589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2357b0-6653-49ea-889e-63610f414ad8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:17.629116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ca2357b0-6653-49ea-889e-63610f414ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:17.773241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2357b0-6653-49ea-889e-63610f414ad8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:17.800967 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:17.885476 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6ea71-cf00-4f55-91b3-61bc0b07f064 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:17.885993 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6ea71-cf00-4f55-91b3-61bc0b07f064 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:17.886299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6ea71-cf00-4f55-91b3-61bc0b07f064 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:17.886569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6ea71-cf00-4f55-91b3-61bc0b07f064 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:17.888147 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-76c6ea71-cf00-4f55-91b3-61bc0b07f064 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:17.979066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6ea71-cf00-4f55-91b3-61bc0b07f064 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:17.980984 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c6ea71-cf00-4f55-91b3-61bc0b07f064 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:17.998727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 723/1440] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:20.091994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308bdce-0dab-4de0-8c89-34c806ff2da9 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:20.092555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308bdce-0dab-4de0-8c89-34c806ff2da9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:20.092555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308bdce-0dab-4de0-8c89-34c806ff2da9 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:20.094462 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:18:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:20.099835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3a5d5-cee8-40a4-a9bd-209a0356c615 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:20.100272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3a5d5-cee8-40a4-a9bd-209a0356c615 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:20.100465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3a5d5-cee8-40a4-a9bd-209a0356c615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:20.140632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:20.166700 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-38d3a5d5-cee8-40a4-a9bd-209a0356c615 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:20.167903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-38d3a5d5-cee8-40a4-a9bd-209a0356c615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:20.265866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:18:20.548920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308bdce-0dab-4de0-8c89-34c806ff2da9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:20.550474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5308bdce-0dab-4de0-8c89-34c806ff2da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:20.658409 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308bdce-0dab-4de0-8c89-34c806ff2da9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:20.690334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:20.732037 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae85dc-1937-42d3-b74c-be8ba63dffa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:20.732629 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae85dc-1937-42d3-b74c-be8ba63dffa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:20.732898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae85dc-1937-42d3-b74c-be8ba63dffa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:20.733159 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae85dc-1937-42d3-b74c-be8ba63dffa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:20.734949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d7ae85dc-1937-42d3-b74c-be8ba63dffa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:20.840943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae85dc-1937-42d3-b74c-be8ba63dffa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:20.842299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae85dc-1937-42d3-b74c-be8ba63dffa7 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:20.860460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 719/1443] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:23.222239 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea09d19-3607-4b2b-b9b3-5554529b69e1 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:23.222814 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea09d19-3607-4b2b-b9b3-5554529b69e1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:23.222814 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea09d19-3607-4b2b-b9b3-5554529b69e1 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:23.224127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:23.229861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994506cf-10d6-4b6a-8fe4-e974f63e6a1c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:23.230340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994506cf-10d6-4b6a-8fe4-e974f63e6a1c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:23.230446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994506cf-10d6-4b6a-8fe4-e974f63e6a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:23.295382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-994506cf-10d6-4b6a-8fe4-e974f63e6a1c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:23.296606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-994506cf-10d6-4b6a-8fe4-e974f63e6a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:23.309539 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:23.393534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:18:23.679285 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea09d19-3607-4b2b-b9b3-5554529b69e1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:23.680857 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-aea09d19-3607-4b2b-b9b3-5554529b69e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:23.784063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea09d19-3607-4b2b-b9b3-5554529b69e1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:23.797086 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:23.858042 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56023cc5-2d56-46b1-a1f2-70f17a55cc51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:23.858673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56023cc5-2d56-46b1-a1f2-70f17a55cc51 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:23.858950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56023cc5-2d56-46b1-a1f2-70f17a55cc51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:23.859222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56023cc5-2d56-46b1-a1f2-70f17a55cc51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:23.860824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-56023cc5-2d56-46b1-a1f2-70f17a55cc51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:23.969426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56023cc5-2d56-46b1-a1f2-70f17a55cc51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:23.970877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56023cc5-2d56-46b1-a1f2-70f17a55cc51 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:23.991153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 726/1446] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:18:26.439815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2c3d-4f47-4924-90da-aedbc683da8c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:26.440326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2c3d-4f47-4924-90da-aedbc683da8c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:26.440326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e2c3d-4f47-4924-90da-aedbc683da8c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:26.441619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:26.447279 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4963f15-3ddc-4261-bd08-db506771d811 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:26.447678 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4963f15-3ddc-4261-bd08-db506771d811 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:26.447827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4963f15-3ddc-4261-bd08-db506771d811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:26.495755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:26.514206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e4963f15-3ddc-4261-bd08-db506771d811 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:26.516013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e4963f15-3ddc-4261-bd08-db506771d811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:26.610788 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:26.897423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e2c3d-4f47-4924-90da-aedbc683da8c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:26.898983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-905e2c3d-4f47-4924-90da-aedbc683da8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:27.002502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e2c3d-4f47-4924-90da-aedbc683da8c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:27.043103 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:27.076280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32c7cd-8b52-4b1d-9a40-c65c6f9089b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:27.076870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32c7cd-8b52-4b1d-9a40-c65c6f9089b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:27.077671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32c7cd-8b52-4b1d-9a40-c65c6f9089b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:27.077941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32c7cd-8b52-4b1d-9a40-c65c6f9089b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:27.079552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-df32c7cd-8b52-4b1d-9a40-c65c6f9089b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:27.188683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32c7cd-8b52-4b1d-9a40-c65c6f9089b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:27.190038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32c7cd-8b52-4b1d-9a40-c65c6f9089b9 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:27.208291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 722/1449] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:29.483252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1597516-9b88-4c64-8b06-84a20b16b379 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:29.484347 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1597516-9b88-4c64-8b06-84a20b16b379 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:29.484453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1597516-9b88-4c64-8b06-84a20b16b379 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:29.485757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:18:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:29.491279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845006e-2395-4940-a7a8-80cb8d095086 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:29.491688 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845006e-2395-4940-a7a8-80cb8d095086 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:29.491827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845006e-2395-4940-a7a8-80cb8d095086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:29.558775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1845006e-2395-4940-a7a8-80cb8d095086 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:29.559967 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1845006e-2395-4940-a7a8-80cb8d095086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:29.579879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:29.654365 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:29.937971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1597516-9b88-4c64-8b06-84a20b16b379 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:29.939594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f1597516-9b88-4c64-8b06-84a20b16b379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:30.043275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1597516-9b88-4c64-8b06-84a20b16b379 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:30.064212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:30.116042 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20175b92-7220-45a3-a8f2-274ae6a3e2b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:30.116654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20175b92-7220-45a3-a8f2-274ae6a3e2b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:30.116924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20175b92-7220-45a3-a8f2-274ae6a3e2b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:30.117186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20175b92-7220-45a3-a8f2-274ae6a3e2b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:30.118782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-20175b92-7220-45a3-a8f2-274ae6a3e2b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:30.227829 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20175b92-7220-45a3-a8f2-274ae6a3e2b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:30.229483 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20175b92-7220-45a3-a8f2-274ae6a3e2b6 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:30.248292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 729/1452] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:32.485250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4089dab-db1d-4bae-83d5-0877b2b401ee None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:32.485784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4089dab-db1d-4bae-83d5-0877b2b401ee None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:32.485983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4089dab-db1d-4bae-83d5-0877b2b401ee None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:32.487683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:18:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:32.493355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d64ec-3254-4a6d-89e0-eec7ee1790f3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:32.493753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d64ec-3254-4a6d-89e0-eec7ee1790f3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:32.494008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d64ec-3254-4a6d-89e0-eec7ee1790f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:32.559124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:32.562217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2e2d64ec-3254-4a6d-89e0-eec7ee1790f3 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:32.564373 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2e2d64ec-3254-4a6d-89e0-eec7ee1790f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:32.672478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:32.963666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4089dab-db1d-4bae-83d5-0877b2b401ee None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:32.965245 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e4089dab-db1d-4bae-83d5-0877b2b401ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:33.073677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4089dab-db1d-4bae-83d5-0877b2b401ee None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:33.122645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:33.146692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac7395-037b-4734-a3f3-15b76f357de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:33.147284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac7395-037b-4734-a3f3-15b76f357de4 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:33.147554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac7395-037b-4734-a3f3-15b76f357de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:33.147818 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac7395-037b-4734-a3f3-15b76f357de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:33.149382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7fac7395-037b-4734-a3f3-15b76f357de4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:33.258728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac7395-037b-4734-a3f3-15b76f357de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:33.260078 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac7395-037b-4734-a3f3-15b76f357de4 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:33.278261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 725/1455] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:35.509758 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa07eb-6fdf-4dab-a39c-5e609b22015b None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:35.510481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa07eb-6fdf-4dab-a39c-5e609b22015b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:35.510481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa07eb-6fdf-4dab-a39c-5e609b22015b None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:35.511692 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:35.517652 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf5441-d3fb-4ad6-a9fa-98003386e3c2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:35.518100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf5441-d3fb-4ad6-a9fa-98003386e3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:35.518219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf5441-d3fb-4ad6-a9fa-98003386e3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:35.583631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-67bf5441-d3fb-4ad6-a9fa-98003386e3c2 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:35.584763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-67bf5441-d3fb-4ad6-a9fa-98003386e3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:35.613324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:35.685106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:35.974486 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa07eb-6fdf-4dab-a39c-5e609b22015b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:35.976064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-88fa07eb-6fdf-4dab-a39c-5e609b22015b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:36.080026 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa07eb-6fdf-4dab-a39c-5e609b22015b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:36.107642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:36.152738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd4071-843f-4bb3-a96d-b2d719936725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:36.153363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd4071-843f-4bb3-a96d-b2d719936725 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:36.153638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd4071-843f-4bb3-a96d-b2d719936725 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:36.153907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd4071-843f-4bb3-a96d-b2d719936725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:36.156316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-90fd4071-843f-4bb3-a96d-b2d719936725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:36.267624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd4071-843f-4bb3-a96d-b2d719936725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:36.268950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fd4071-843f-4bb3-a96d-b2d719936725 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:36.288890 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 732/1458] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:38.476931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6545852-949a-4187-af03-c6061d90733a None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:38.477491 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6545852-949a-4187-af03-c6061d90733a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:38.477491 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6545852-949a-4187-af03-c6061d90733a None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:38.479299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:18:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:38.485350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096b692-9422-472b-9565-4dffa384362d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:38.485777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096b692-9422-472b-9565-4dffa384362d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:38.485894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096b692-9422-472b-9565-4dffa384362d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:38.550913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2096b692-9422-472b-9565-4dffa384362d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:38.552122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2096b692-9422-472b-9565-4dffa384362d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:38.560304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:38.647015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:38.931841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6545852-949a-4187-af03-c6061d90733a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:38.933409 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d6545852-949a-4187-af03-c6061d90733a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:39.036403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6545852-949a-4187-af03-c6061d90733a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:39.090662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:39.109192 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f810c4-7c55-4029-bb98-05322a04ffa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:39.109782 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f810c4-7c55-4029-bb98-05322a04ffa1 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:39.110049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f810c4-7c55-4029-bb98-05322a04ffa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:39.110345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f810c4-7c55-4029-bb98-05322a04ffa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:39.111914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-09f810c4-7c55-4029-bb98-05322a04ffa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:39.219383 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f810c4-7c55-4029-bb98-05322a04ffa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:39.220730 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f810c4-7c55-4029-bb98-05322a04ffa1 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:39.239081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 728/1461] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:41.445081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e9e55-c01b-4da8-a8e9-a9762f904f5d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:41.445608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e9e55-c01b-4da8-a8e9-a9762f904f5d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:41.445608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e9e55-c01b-4da8-a8e9-a9762f904f5d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:41.446996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:41.452259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a7b90-c4fd-41d2-8539-0a1a352d401d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:41.452703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a7b90-c4fd-41d2-8539-0a1a352d401d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:41.452800 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a7b90-c4fd-41d2-8539-0a1a352d401d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:41.516824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2c5a7b90-c4fd-41d2-8539-0a1a352d401d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:41.518015 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2c5a7b90-c4fd-41d2-8539-0a1a352d401d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:41.555661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:41.613382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:41.904308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e9e55-c01b-4da8-a8e9-a9762f904f5d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:41.906484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0b2e9e55-c01b-4da8-a8e9-a9762f904f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:42.007333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e9e55-c01b-4da8-a8e9-a9762f904f5d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:42.044974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:42.080167 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f0605-b5de-4c99-bfaa-4a0f94e9c2d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:42.080762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f0605-b5de-4c99-bfaa-4a0f94e9c2d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:42.081043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f0605-b5de-4c99-bfaa-4a0f94e9c2d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:42.081319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f0605-b5de-4c99-bfaa-4a0f94e9c2d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:42.082939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cc3f0605-b5de-4c99-bfaa-4a0f94e9c2d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:42.193032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f0605-b5de-4c99-bfaa-4a0f94e9c2d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:42.194492 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f0605-b5de-4c99-bfaa-4a0f94e9c2d1 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:42.213043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 735/1464] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:44.393342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636065af-ae06-426e-87ba-54ff42b1266f None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:44.393868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636065af-ae06-426e-87ba-54ff42b1266f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:44.393868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636065af-ae06-426e-87ba-54ff42b1266f None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:44.395251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:18:44.400805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a61cc-02ea-4a03-9f2a-21b49852d545 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:44.401232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a61cc-02ea-4a03-9f2a-21b49852d545 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:44.401365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a61cc-02ea-4a03-9f2a-21b49852d545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:44.466101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-601a61cc-02ea-4a03-9f2a-21b49852d545 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:44.467230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-601a61cc-02ea-4a03-9f2a-21b49852d545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:44.485049 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:44.562388 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:44.846332 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636065af-ae06-426e-87ba-54ff42b1266f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:44.847870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-636065af-ae06-426e-87ba-54ff42b1266f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:44.956155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636065af-ae06-426e-87ba-54ff42b1266f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:45.022267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:45.029202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344b15e-01f5-4110-8b02-d02362317e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:45.030412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344b15e-01f5-4110-8b02-d02362317e6d service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:45.030689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344b15e-01f5-4110-8b02-d02362317e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:45.030954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344b15e-01f5-4110-8b02-d02362317e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:45.032528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0344b15e-01f5-4110-8b02-d02362317e6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:45.148571 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344b15e-01f5-4110-8b02-d02362317e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:45.150009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344b15e-01f5-4110-8b02-d02362317e6d service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:45.172626 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 731/1467] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:47.346270 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f32544-30ff-4d2f-95b2-54d17718183e None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:47.346725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f32544-30ff-4d2f-95b2-54d17718183e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:47.346823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f32544-30ff-4d2f-95b2-54d17718183e None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:47.348136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:18:47.353818 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b303e6a-e890-4965-a442-00170c844b1b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:47.354275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b303e6a-e890-4965-a442-00170c844b1b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:47.354399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b303e6a-e890-4965-a442-00170c844b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:47.419295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9b303e6a-e890-4965-a442-00170c844b1b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:18:47.421028 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9b303e6a-e890-4965-a442-00170c844b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:47.467963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:47.515458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:18:47.797669 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f32544-30ff-4d2f-95b2-54d17718183e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:18:47.799269 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-83f32544-30ff-4d2f-95b2-54d17718183e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:47.899794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f32544-30ff-4d2f-95b2-54d17718183e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:18:47.945763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:47.972052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab825f66-6fcd-4037-bcb5-8fae76023e9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:18:47.972639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab825f66-6fcd-4037-bcb5-8fae76023e9d service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:18:47.972916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab825f66-6fcd-4037-bcb5-8fae76023e9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:18:47.973429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab825f66-6fcd-4037-bcb5-8fae76023e9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:18:47.975025 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ab825f66-6fcd-4037-bcb5-8fae76023e9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:18:48.087449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab825f66-6fcd-4037-bcb5-8fae76023e9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:18:48.088790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab825f66-6fcd-4037-bcb5-8fae76023e9d service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:18:48.224642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 738/1470] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:18:57.514474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=22 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:18:58.222186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=66 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:08.249422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef589754-5986-47b7-a3dc-eefe4b78a8d1 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:08.249978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef589754-5986-47b7-a3dc-eefe4b78a8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:08.249978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef589754-5986-47b7-a3dc-eefe4b78a8d1 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:08.251354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 733/1471] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:19:08.256480 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d41d23-f656-4cfa-b9f5-9eb40a61dc79 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:08.256917 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d41d23-f656-4cfa-b9f5-9eb40a61dc79 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:08.257029 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d41d23-f656-4cfa-b9f5-9eb40a61dc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:08.322166 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-04d41d23-f656-4cfa-b9f5-9eb40a61dc79 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:19:08.323371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-04d41d23-f656-4cfa-b9f5-9eb40a61dc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:08.417684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 739/1472] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:19:08.719117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb880fa3-3007-4474-85c5-b49c6f3f8d14 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:08.719401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb880fa3-3007-4474-85c5-b49c6f3f8d14 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:08.719676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb880fa3-3007-4474-85c5-b49c6f3f8d14 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:08.721011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 734/1473] 10.222.0.55 () {54 vars in 962 bytes} [Tue Sep 27 17:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:19:08.725967 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d7976-1975-4aba-8e9d-285db51e5acf None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:08.726269 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d7976-1975-4aba-8e9d-285db51e5acf None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:08.726541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d7976-1975-4aba-8e9d-285db51e5acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:08.800867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5c8d7976-1975-4aba-8e9d-285db51e5acf None None] MFA Rules not processed for user `6cc89c1a7e684612a73ec9fef6bda780`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:19:08.802061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5c8d7976-1975-4aba-8e9d-285db51e5acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:08.822411 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:08.877149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 740/1474] 10.222.0.55 () {58 vars in 1059 bytes} [Tue Sep 27 17:19:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:19:08.885031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb880fa3-3007-4474-85c5-b49c6f3f8d14 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:19:08.886601 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-bb880fa3-3007-4474-85c5-b49c6f3f8d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:08.988278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb880fa3-3007-4474-85c5-b49c6f3f8d14 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:19:09.058189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7037e33-ce0b-4bed-8bc3-a35052debfbf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc89c1a7e684612a73ec9fef6bda780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:19:09.058779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7037e33-ce0b-4bed-8bc3-a35052debfbf service glance] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:09.059048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7037e33-ce0b-4bed-8bc3-a35052debfbf service glance] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:09.059310 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7037e33-ce0b-4bed-8bc3-a35052debfbf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:09.061444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c7037e33-ce0b-4bed-8bc3-a35052debfbf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:09.113034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:09.167450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7037e33-ce0b-4bed-8bc3-a35052debfbf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:19:09.168870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7037e33-ce0b-4bed-8bc3-a35052debfbf service glance] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:19:09.187259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 735/1475] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:19:11.424959 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe23d6-01fa-4bff-a293-615aaf97e362 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:11.425533 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe23d6-01fa-4bff-a293-615aaf97e362 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:11.425533 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe23d6-01fa-4bff-a293-615aaf97e362 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:11.426849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 741/1476] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:19:11.432205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619f2d7-32b8-4b42-877f-7f6b22ccdfa9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:11.432646 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619f2d7-32b8-4b42-877f-7f6b22ccdfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:11.432776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619f2d7-32b8-4b42-877f-7f6b22ccdfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:11.497413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9619f2d7-32b8-4b42-877f-7f6b22ccdfa9 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:19:11.499194 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9619f2d7-32b8-4b42-877f-7f6b22ccdfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:11.593760 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 736/1477] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:19:11.891553 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe23d6-01fa-4bff-a293-615aaf97e362 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:19:11.892984 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-17fe23d6-01fa-4bff-a293-615aaf97e362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:12.027351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe23d6-01fa-4bff-a293-615aaf97e362 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:19:12.079802 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:12.124838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0059977-bc7d-47a7-a180-f7291b50f86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:19:12.125451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0059977-bc7d-47a7-a180-f7291b50f86b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:12.125717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0059977-bc7d-47a7-a180-f7291b50f86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:12.125975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0059977-bc7d-47a7-a180-f7291b50f86b admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:12.126626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0059977-bc7d-47a7-a180-f7291b50f86b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:19:12.128343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-f0059977-bc7d-47a7-a180-f7291b50f86b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:19:12.128343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 27 17:19:12.128343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: Sep 27 17:19:12.129452 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0059977-bc7d-47a7-a180-f7291b50f86b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:19:12.148057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 742/1478] 10.222.0.55 () {56 vars in 1108 bytes} [Tue Sep 27 17:19:11 2022] GET /identity/v3/projects => generated 2553 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:19:14.322654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ae911-7c08-4720-8a4f-774004934405 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:14.323116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ae911-7c08-4720-8a4f-774004934405 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:14.323237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ae911-7c08-4720-8a4f-774004934405 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:14.324573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 737/1479] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:19:14.330297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577a718-44c3-4ee2-8d15-a21756c4bae2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:14.330735 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577a718-44c3-4ee2-8d15-a21756c4bae2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:14.330844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577a718-44c3-4ee2-8d15-a21756c4bae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:14.400800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e577a718-44c3-4ee2-8d15-a21756c4bae2 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:19:14.402011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e577a718-44c3-4ee2-8d15-a21756c4bae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:14.494868 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 743/1480] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:19:14.790747 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ae911-7c08-4720-8a4f-774004934405 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:19:14.792296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ca7ae911-7c08-4720-8a4f-774004934405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:14.802456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:14.935345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ae911-7c08-4720-8a4f-774004934405 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:19:15.015732 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:15.048167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa0fd6-f64c-4b10-8fdf-c1ccb947ccbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:19:15.048704 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa0fd6-f64c-4b10-8fdf-c1ccb947ccbf service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:15.048976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa0fd6-f64c-4b10-8fdf-c1ccb947ccbf service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:15.049261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa0fd6-f64c-4b10-8fdf-c1ccb947ccbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:15.050868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6caa0fd6-f64c-4b10-8fdf-c1ccb947ccbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:15.139790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa0fd6-f64c-4b10-8fdf-c1ccb947ccbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:19:15.141373 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa0fd6-f64c-4b10-8fdf-c1ccb947ccbf service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:19:15.158900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 738/1481] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:19:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:19:17.487234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1900d-6a4e-4781-bc65-8bb3cc3e3b03 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:17.487808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1900d-6a4e-4781-bc65-8bb3cc3e3b03 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:17.487808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1900d-6a4e-4781-bc65-8bb3cc3e3b03 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:17.489115 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 744/1482] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:19:17.494403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d499c2-ca2e-41b0-a131-8200b4baa697 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:17.494833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d499c2-ca2e-41b0-a131-8200b4baa697 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:17.494950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d499c2-ca2e-41b0-a131-8200b4baa697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:17.559522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c2d499c2-ca2e-41b0-a131-8200b4baa697 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:19:17.560756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c2d499c2-ca2e-41b0-a131-8200b4baa697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:17.605578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:17.654834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 739/1483] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:19:17.946499 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1900d-6a4e-4781-bc65-8bb3cc3e3b03 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:19:17.948059 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-79d1900d-6a4e-4781-bc65-8bb3cc3e3b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:17.972635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:18.050231 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1900d-6a4e-4781-bc65-8bb3cc3e3b03 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:19:18.121998 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170a4aa-6e5d-4c38-854e-4a106818a389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:19:18.123152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170a4aa-6e5d-4c38-854e-4a106818a389 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:18.123425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170a4aa-6e5d-4c38-854e-4a106818a389 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:18.123690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170a4aa-6e5d-4c38-854e-4a106818a389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:18.125255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d170a4aa-6e5d-4c38-854e-4a106818a389 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:18.196298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:18.233756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170a4aa-6e5d-4c38-854e-4a106818a389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:19:18.235257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170a4aa-6e5d-4c38-854e-4a106818a389 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:19:18.253716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 745/1484] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:19:20.462121 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34000ff3-4cc1-4d6c-a503-5e1184cfab5c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:20.462654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34000ff3-4cc1-4d6c-a503-5e1184cfab5c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:20.462654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34000ff3-4cc1-4d6c-a503-5e1184cfab5c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:20.463961 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 740/1485] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:19:20.469308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f174bf-e085-4cfb-8374-1d618423496e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:20.469687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f174bf-e085-4cfb-8374-1d618423496e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:20.469886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f174bf-e085-4cfb-8374-1d618423496e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:20.535027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e9f174bf-e085-4cfb-8374-1d618423496e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:19:20.536244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e9f174bf-e085-4cfb-8374-1d618423496e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:20.630862 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 746/1486] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:19:20.919903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34000ff3-4cc1-4d6c-a503-5e1184cfab5c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:19:20.921431 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-34000ff3-4cc1-4d6c-a503-5e1184cfab5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:21.027943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34000ff3-4cc1-4d6c-a503-5e1184cfab5c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:19:21.098545 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:21.100107 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482fa2d-4520-4092-9ed6-13671a156fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:19:21.100746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482fa2d-4520-4092-9ed6-13671a156fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:21.101067 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482fa2d-4520-4092-9ed6-13671a156fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:21.101342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482fa2d-4520-4092-9ed6-13671a156fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:21.103494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a482fa2d-4520-4092-9ed6-13671a156fe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:21.211562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482fa2d-4520-4092-9ed6-13671a156fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:19:21.212947 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482fa2d-4520-4092-9ed6-13671a156fe5 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:19:21.231387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 741/1487] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:19:23.529680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab4b18-2372-43f3-b7a1-f34d0b76152b None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:23.530277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab4b18-2372-43f3-b7a1-f34d0b76152b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:23.530277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab4b18-2372-43f3-b7a1-f34d0b76152b None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:23.531607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 747/1488] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:19:23.536856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbc829-b31e-4fcd-9227-7f2a2ff99c1d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:23.537259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbc829-b31e-4fcd-9227-7f2a2ff99c1d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:23.537402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cbc829-b31e-4fcd-9227-7f2a2ff99c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:23.602571 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e7cbc829-b31e-4fcd-9227-7f2a2ff99c1d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:19:23.604013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e7cbc829-b31e-4fcd-9227-7f2a2ff99c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:23.656409 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:23.699859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 742/1489] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:19:23.994904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab4b18-2372-43f3-b7a1-f34d0b76152b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:19:23.996459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-73ab4b18-2372-43f3-b7a1-f34d0b76152b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:24.032266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:24.098582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab4b18-2372-43f3-b7a1-f34d0b76152b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:19:24.171964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c614c-2fdc-4306-9baa-eaa56527d64f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:19:24.172632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c614c-2fdc-4306-9baa-eaa56527d64f service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:24.172906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c614c-2fdc-4306-9baa-eaa56527d64f service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:24.173169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c614c-2fdc-4306-9baa-eaa56527d64f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:24.174899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ef7c614c-2fdc-4306-9baa-eaa56527d64f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:24.254509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:24.283743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c614c-2fdc-4306-9baa-eaa56527d64f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:19:24.285168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c614c-2fdc-4306-9baa-eaa56527d64f service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:19:24.303919 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 748/1490] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:19:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:19:26.663843 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d256110-8d30-4f29-8bcf-9240d2d66118 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:26.664447 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d256110-8d30-4f29-8bcf-9240d2d66118 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:26.664447 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d256110-8d30-4f29-8bcf-9240d2d66118 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:26.665732 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 743/1491] 10.222.0.55 () {54 vars in 927 bytes} [Tue Sep 27 17:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:19:26.671646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6f9ed-c9d4-45ee-ac39-c569d76e1a2b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:26.672084 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6f9ed-c9d4-45ee-ac39-c569d76e1a2b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:26.672189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6f9ed-c9d4-45ee-ac39-c569d76e1a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:26.737285 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e0d6f9ed-c9d4-45ee-ac39-c569d76e1a2b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:19:26.738531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e0d6f9ed-c9d4-45ee-ac39-c569d76e1a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:26.835518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 749/1492] 10.222.0.55 () {58 vars in 1024 bytes} [Tue Sep 27 17:19:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:19:27.131899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d256110-8d30-4f29-8bcf-9240d2d66118 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:19:27.133436 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9d256110-8d30-4f29-8bcf-9240d2d66118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:27.235908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d256110-8d30-4f29-8bcf-9240d2d66118 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:19:27.307615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e579f4-ad26-4fc5-8177-b5c8cb3439c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:19:27.308213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e579f4-ad26-4fc5-8177-b5c8cb3439c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:19:27.308490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e579f4-ad26-4fc5-8177-b5c8cb3439c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:19:27.308758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e579f4-ad26-4fc5-8177-b5c8cb3439c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:19:27.310422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-48e579f4-ad26-4fc5-8177-b5c8cb3439c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:19:27.319158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:27.417997 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e579f4-ad26-4fc5-8177-b5c8cb3439c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:19:27.419954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e579f4-ad26-4fc5-8177-b5c8cb3439c7 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:19:27.437401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 744/1493] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:19:36.833978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=82 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:19:37.435510 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=50 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:20:39.877454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a257c2-ee53-454e-b5d5-6f7894d90721 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:39.878013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a257c2-ee53-454e-b5d5-6f7894d90721 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:39.878013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a257c2-ee53-454e-b5d5-6f7894d90721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:39.956871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c4a257c2-ee53-454e-b5d5-6f7894d90721 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:20:39.958116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c4a257c2-ee53-454e-b5d5-6f7894d90721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:40.053594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 750/1494] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:20:40.061915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa58fa7-6a73-4d3e-b5bf-ba7785e99713 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:40.062430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa58fa7-6a73-4d3e-b5bf-ba7785e99713 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:40.062552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa58fa7-6a73-4d3e-b5bf-ba7785e99713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:40.138206 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2aa58fa7-6a73-4d3e-b5bf-ba7785e99713 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:20:40.139374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2aa58fa7-6a73-4d3e-b5bf-ba7785e99713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:40.239836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 745/1495] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:20:40.244430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a257c2-ee53-454e-b5d5-6f7894d90721 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:20:40.246151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c4a257c2-ee53-454e-b5d5-6f7894d90721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:40.257442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a257c2-ee53-454e-b5d5-6f7894d90721 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:20:40.320399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:20:40.360208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d3bc0-8de1-49fc-9765-ea6908438b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:20:40.360805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d3bc0-8de1-49fc-9765-ea6908438b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:40.361078 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d3bc0-8de1-49fc-9765-ea6908438b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:40.361342 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d3bc0-8de1-49fc-9765-ea6908438b0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:40.361981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d3bc0-8de1-49fc-9765-ea6908438b0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:20:40.362304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d3bc0-8de1-49fc-9765-ea6908438b0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:20:40.363305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-9f0d3bc0-8de1-49fc-9765-ea6908438b0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:20:40.363305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 27 17:20:40.363305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: Sep 27 17:20:40.364182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d3bc0-8de1-49fc-9765-ea6908438b0b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:20:40.375841 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 751/1496] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:20:40.384220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa58fa7-6a73-4d3e-b5bf-ba7785e99713 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:20:40.393404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa58fa7-6a73-4d3e-b5bf-ba7785e99713 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:20:40.453353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:20:40.487240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889bf85-2993-4a3f-b139-e6409ce8d291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:20:40.487840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889bf85-2993-4a3f-b139-e6409ce8d291 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:40.488113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889bf85-2993-4a3f-b139-e6409ce8d291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:40.488382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889bf85-2993-4a3f-b139-e6409ce8d291 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:40.489044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889bf85-2993-4a3f-b139-e6409ce8d291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:20:40.490844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-4889bf85-2993-4a3f-b139-e6409ce8d291 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:20:40.490844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 27 17:20:40.490844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: Sep 27 17:20:40.492037 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889bf85-2993-4a3f-b139-e6409ce8d291 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:20:40.547221 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 746/1497] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:20:40 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:20:40.551445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d3bc0-8de1-49fc-9765-ea6908438b0b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:20:40.553025 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9f0d3bc0-8de1-49fc-9765-ea6908438b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:40.586033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d3bc0-8de1-49fc-9765-ea6908438b0b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:20:40.680146 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338078e-0bb6-431b-8206-c5b2deffa57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:20:40.680701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338078e-0bb6-431b-8206-c5b2deffa57f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:40.680975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338078e-0bb6-431b-8206-c5b2deffa57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:40.681242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338078e-0bb6-431b-8206-c5b2deffa57f admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:40.681877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338078e-0bb6-431b-8206-c5b2deffa57f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:20:40.682912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-d338078e-0bb6-431b-8206-c5b2deffa57f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:20:40.682912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 27 17:20:40.682912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: Sep 27 17:20:40.684109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338078e-0bb6-431b-8206-c5b2deffa57f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:20:40.726581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 752/1498] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:20:40 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:20:40.731633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889bf85-2993-4a3f-b139-e6409ce8d291 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:20:40.733130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4889bf85-2993-4a3f-b139-e6409ce8d291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:40.797355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889bf85-2993-4a3f-b139-e6409ce8d291 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:20:40.879928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:20:40.891427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9617c-9a8b-4278-9a88-f18a6a631aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:20:40.892030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9617c-9a8b-4278-9a88-f18a6a631aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:40.892305 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9617c-9a8b-4278-9a88-f18a6a631aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:40.892574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9617c-9a8b-4278-9a88-f18a6a631aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:40.893211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9617c-9a8b-4278-9a88-f18a6a631aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:20:40.894259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-ccc9617c-9a8b-4278-9a88-f18a6a631aa8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:20:40.894259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 27 17:20:40.894259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: Sep 27 17:20:40.895141 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9617c-9a8b-4278-9a88-f18a6a631aa8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:20:40.909606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 747/1499] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:20:40.913793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338078e-0bb6-431b-8206-c5b2deffa57f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:20:40.915337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d338078e-0bb6-431b-8206-c5b2deffa57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:40.927228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338078e-0bb6-431b-8206-c5b2deffa57f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:20:40.951976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:20:41.022912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e1ea96-6601-4350-83bf-c753aadef816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:20:41.023528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1ea96-6601-4350-83bf-c753aadef816 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:41.023799 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1ea96-6601-4350-83bf-c753aadef816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:41.024062 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e1ea96-6601-4350-83bf-c753aadef816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:41.024718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e1ea96-6601-4350-83bf-c753aadef816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:20:41.025717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-c7e1ea96-6601-4350-83bf-c753aadef816 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:20:41.025717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 27 17:20:41.025717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: Sep 27 17:20:41.026694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e1ea96-6601-4350-83bf-c753aadef816 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:20:41.039175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 753/1500] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:20:41.043802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9617c-9a8b-4278-9a88-f18a6a631aa8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:20:41.045311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ccc9617c-9a8b-4278-9a88-f18a6a631aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:41.078611 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9617c-9a8b-4278-9a88-f18a6a631aa8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:20:41.170089 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad765e-a9f7-4f2a-a72d-9a4262bf7a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:20:41.170756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad765e-a9f7-4f2a-a72d-9a4262bf7a7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:41.171033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad765e-a9f7-4f2a-a72d-9a4262bf7a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:41.171304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad765e-a9f7-4f2a-a72d-9a4262bf7a7a admin admin] PATH_INFO: `/v3/projects/234a7189a6bd4e70876b402f62025d51/users/ecfaa0c99ef246538c2c973d1a35fff7/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:41.193564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad765e-a9f7-4f2a-a72d-9a4262bf7a7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=234a7189a6bd4e70876b402f62025d51, user_id=ecfaa0c99ef246538c2c973d1a35fff7, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:20:41.194619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-a3ad765e-a9f7-4f2a-a72d-9a4262bf7a7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:20:41.194619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 27 17:20:41.194619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: Sep 27 17:20:41.196548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad765e-a9f7-4f2a-a72d-9a4262bf7a7a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:20:41.220083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 748/1501] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:20:41 2022] PUT /identity/v3/projects/234a7189a6bd4e70876b402f62025d51/users/ecfaa0c99ef246538c2c973d1a35fff7/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:20:41.232783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319b765-7037-416d-93db-f193d58a12a7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:41.233074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319b765-7037-416d-93db-f193d58a12a7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:41.233348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319b765-7037-416d-93db-f193d58a12a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:41.289922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5319b765-7037-416d-93db-f193d58a12a7 None None] MFA Rules not processed for user `ecfaa0c99ef246538c2c973d1a35fff7`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:20:41.291158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5319b765-7037-416d-93db-f193d58a12a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:41.371694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 754/1502] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:20:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:20:41.384991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def375be-311f-481a-b064-5ab0e0015f1b None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:41.385277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def375be-311f-481a-b064-5ab0e0015f1b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:41.385550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def375be-311f-481a-b064-5ab0e0015f1b None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:41.386931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 749/1503] 10.222.0.55 () {54 vars in 963 bytes} [Tue Sep 27 17:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:20:41.391953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc1515-c6a1-4db5-8b48-16cb667f27ef None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:41.392261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc1515-c6a1-4db5-8b48-16cb667f27ef None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:41.392531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc1515-c6a1-4db5-8b48-16cb667f27ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:41.425526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:20:41.468541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f0bc1515-c6a1-4db5-8b48-16cb667f27ef None None] MFA Rules not processed for user `528981e41f0b436b82989d62c713d993`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:20:41.469727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f0bc1515-c6a1-4db5-8b48-16cb667f27ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:41.543444 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 755/1504] 10.222.0.55 () {58 vars in 1060 bytes} [Tue Sep 27 17:20:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:20:41.550990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def375be-311f-481a-b064-5ab0e0015f1b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:20:41.552571 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-def375be-311f-481a-b064-5ab0e0015f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:41.601862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:20:41.655494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def375be-311f-481a-b064-5ab0e0015f1b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:20:41.730866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ee6db-9975-4573-81e7-742392bc5723 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:20:41.731476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ee6db-9975-4573-81e7-742392bc5723 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:41.731752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ee6db-9975-4573-81e7-742392bc5723 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:41.732024 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ee6db-9975-4573-81e7-742392bc5723 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:41.733617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e42ee6db-9975-4573-81e7-742392bc5723 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:41.837482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ee6db-9975-4573-81e7-742392bc5723 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:20:41.839556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ee6db-9975-4573-81e7-742392bc5723 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:20:41.843057 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:20:41.858039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 750/1505] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:20:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:20:41.903560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95500148-b6ec-4f01-b519-18026872dd7b None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:41.903842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95500148-b6ec-4f01-b519-18026872dd7b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:41.904113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95500148-b6ec-4f01-b519-18026872dd7b None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:41.905406 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 756/1506] 10.222.0.55 () {52 vars in 830 bytes} [Tue Sep 27 17:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:20:41.908974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ee6db-9975-4573-81e7-742392bc5723 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:20:41.910199 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e42ee6db-9975-4573-81e7-742392bc5723 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:42.007703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ee6db-9975-4573-81e7-742392bc5723 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:20:42.098816 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:20:42.099463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:42.099739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:42.100009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] PATH_INFO: `/v3/users/ecfaa0c99ef246538c2c973d1a35fff7` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:42.112678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecfaa0c99ef246538c2c973d1a35fff7)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:20:42.113696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:20:42.113696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 27 17:20:42.113696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: Sep 27 17:20:42.114998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:20:42.125016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:20:42.180500 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecfaa0c99ef246538c2c973d1a35fff7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:20:42.194203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecfaa0c99ef246538c2c973d1a35fff7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:20:42.194647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecfaa0c99ef246538c2c973d1a35fff7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:20:42.204366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecfaa0c99ef246538c2c973d1a35fff7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:20:42.226319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecfaa0c99ef246538c2c973d1a35fff7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:20:42.232454 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 751/1507] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:20:41 2022] DELETE /identity/v3/users/ecfaa0c99ef246538c2c973d1a35fff7 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:20:42.236802 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb36c45-e04d-4b8c-b769-09d8501194d3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:42.237271 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb36c45-e04d-4b8c-b769-09d8501194d3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:42.237381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb36c45-e04d-4b8c-b769-09d8501194d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:42.293787 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6cb36c45-e04d-4b8c-b769-09d8501194d3 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:20:42.295027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6cb36c45-e04d-4b8c-b769-09d8501194d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:42.372889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:20:42.385083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 757/1508] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:20:42.395278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:20:42.396483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:42.497742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c8b45-53ba-41f8-bf5c-edec2fb61981 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:20:42.551660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:20:42.572779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11879ef8-eafd-4edb-89ab-d804cd23a014 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:20:42.573385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11879ef8-eafd-4edb-89ab-d804cd23a014 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:42.573661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11879ef8-eafd-4edb-89ab-d804cd23a014 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:42.573930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11879ef8-eafd-4edb-89ab-d804cd23a014 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:42.575555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-11879ef8-eafd-4edb-89ab-d804cd23a014 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:42.681429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11879ef8-eafd-4edb-89ab-d804cd23a014 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:20:42.683466 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11879ef8-eafd-4edb-89ab-d804cd23a014 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:20:42.701169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 752/1509] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:20:43.034095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb36c45-e04d-4b8c-b769-09d8501194d3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:20:43.035600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6cb36c45-e04d-4b8c-b769-09d8501194d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:20:43.050664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb36c45-e04d-4b8c-b769-09d8501194d3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:20:43.149277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f1fa6-5740-4415-bcb8-11069ce8cd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:20:43.149904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f1fa6-5740-4415-bcb8-11069ce8cd78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:20:43.150218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f1fa6-5740-4415-bcb8-11069ce8cd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:20:43.150490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f1fa6-5740-4415-bcb8-11069ce8cd78 admin admin] PATH_INFO: `/v3/projects/234a7189a6bd4e70876b402f62025d51` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:20:43.162120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f1fa6-5740-4415-bcb8-11069ce8cd78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=234a7189a6bd4e70876b402f62025d51)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:20:43.163155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-e39f1fa6-5740-4415-bcb8-11069ce8cd78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:20:43.163155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 27 17:20:43.163155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: Sep 27 17:20:43.164356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f1fa6-5740-4415-bcb8-11069ce8cd78 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:20:43.226597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e39f1fa6-5740-4415-bcb8-11069ce8cd78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '234a7189a6bd4e70876b402f62025d51', 'request_id': 'req-e39f1fa6-5740-4415-bcb8-11069ce8cd78'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:20:43.233395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e39f1fa6-5740-4415-bcb8-11069ce8cd78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '234a7189a6bd4e70876b402f62025d51', 'request_id': 'req-e39f1fa6-5740-4415-bcb8-11069ce8cd78'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:20:43.246343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e39f1fa6-5740-4415-bcb8-11069ce8cd78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '234a7189a6bd4e70876b402f62025d51', 'request_id': 'req-e39f1fa6-5740-4415-bcb8-11069ce8cd78'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:20:43.273099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e39f1fa6-5740-4415-bcb8-11069ce8cd78 admin admin] The token cache is being invalidate because project 234a7189a6bd4e70876b402f62025d51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:20:43.273389 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e39f1fa6-5740-4415-bcb8-11069ce8cd78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:20:43.273812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e39f1fa6-5740-4415-bcb8-11069ce8cd78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:20:43.275229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 758/1510] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:20:43 2022] DELETE /identity/v3/projects/234a7189a6bd4e70876b402f62025d51 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:20:52.698959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=62 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:20:53.271053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:22:07.509398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34342e28-0bc2-4abb-a9d6-10f4a7dcd739 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:07.509963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34342e28-0bc2-4abb-a9d6-10f4a7dcd739 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:07.509963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34342e28-0bc2-4abb-a9d6-10f4a7dcd739 None None] PATH_INFO: `/v3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:07.511879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 753/1511] 10.222.0.18 () {54 vars in 925 bytes} [Tue Sep 27 17:22:07 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:22:07.518910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547c3a1-de0d-43e7-ba8d-4dcdb4afe36b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:07.519339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547c3a1-de0d-43e7-ba8d-4dcdb4afe36b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:07.519447 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547c3a1-de0d-43e7-ba8d-4dcdb4afe36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:07.599954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8547c3a1-de0d-43e7-ba8d-4dcdb4afe36b None None] MFA Rules not processed for user `3b16c430b870424c8d366c9a2959003e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:22:07.601162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8547c3a1-de0d-43e7-ba8d-4dcdb4afe36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:07.714497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 759/1512] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Sep 27 17:22:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:22:09.421902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f61e6c-76bc-416c-a34d-ba01f2dd6b0e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:09.422453 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f61e6c-76bc-416c-a34d-ba01f2dd6b0e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:09.422453 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f61e6c-76bc-416c-a34d-ba01f2dd6b0e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:09.423801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 754/1513] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 17:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:22:09.428712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f99fa-6132-41c7-b7ec-5bcc5772ad0b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:09.429111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f99fa-6132-41c7-b7ec-5bcc5772ad0b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:09.429261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f99fa-6132-41c7-b7ec-5bcc5772ad0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:09.488375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9e8f99fa-6132-41c7-b7ec-5bcc5772ad0b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:22:09.489581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9e8f99fa-6132-41c7-b7ec-5bcc5772ad0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:09.509916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:22:09.574968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 760/1514] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 17:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:22:09.727260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f61e6c-76bc-416c-a34d-ba01f2dd6b0e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:22:09.729806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-88f61e6c-76bc-416c-a34d-ba01f2dd6b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:09.831600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f61e6c-76bc-416c-a34d-ba01f2dd6b0e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:22:09.836911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c66a95-7f94-4035-8c5e-bf569779d392 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:22:09.837509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c66a95-7f94-4035-8c5e-bf569779d392 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:09.837787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c66a95-7f94-4035-8c5e-bf569779d392 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:09.838058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c66a95-7f94-4035-8c5e-bf569779d392 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:09.845969 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c66a95-7f94-4035-8c5e-bf569779d392 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:22:09.847565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c66a95-7f94-4035-8c5e-bf569779d392 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:22:09.856958 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 755/1515] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:22:10.979033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291de08-b506-45bc-96c4-1907a03fbbe6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:10.979875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291de08-b506-45bc-96c4-1907a03fbbe6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:10.979875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291de08-b506-45bc-96c4-1907a03fbbe6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:10.981224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 761/1516] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 17:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:22:10.987355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c43224-1f36-4ac9-a769-93e3803458a7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:10.987789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c43224-1f36-4ac9-a769-93e3803458a7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:10.987899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c43224-1f36-4ac9-a769-93e3803458a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:11.006863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-94c43224-1f36-4ac9-a769-93e3803458a7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:22:11.008034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-94c43224-1f36-4ac9-a769-93e3803458a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:11.016200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 756/1517] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 17:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:22:11.166979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291de08-b506-45bc-96c4-1907a03fbbe6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:22:11.181203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0291de08-b506-45bc-96c4-1907a03fbbe6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:22:11.186532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d1094-7db3-46f0-be0a-630873d5db18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:22:11.187112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d1094-7db3-46f0-be0a-630873d5db18 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:11.187385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d1094-7db3-46f0-be0a-630873d5db18 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:11.187652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d1094-7db3-46f0-be0a-630873d5db18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:11.195516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d1094-7db3-46f0-be0a-630873d5db18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:22:11.197071 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d1094-7db3-46f0-be0a-630873d5db18 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:22:11.206628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 762/1518] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:22:14.561063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311647b4-5542-49a7-9d85-a6dac86ce588 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:14.561497 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311647b4-5542-49a7-9d85-a6dac86ce588 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:14.561603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311647b4-5542-49a7-9d85-a6dac86ce588 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:14.562984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 757/1519] 10.222.0.55 () {54 vars in 961 bytes} [Tue Sep 27 17:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:22:14.568161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bba75-f9a6-43eb-af06-1c4e31df7c29 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:14.568612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bba75-f9a6-43eb-af06-1c4e31df7c29 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:14.568710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bba75-f9a6-43eb-af06-1c4e31df7c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:14.587631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e79bba75-f9a6-43eb-af06-1c4e31df7c29 None None] MFA Rules not processed for user `3b16c430b870424c8d366c9a2959003e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:22:14.588829 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e79bba75-f9a6-43eb-af06-1c4e31df7c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:14.597719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 763/1520] 10.222.0.55 () {58 vars in 1058 bytes} [Tue Sep 27 17:22:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:22:14.606036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311647b4-5542-49a7-9d85-a6dac86ce588 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:22:14.619044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311647b4-5542-49a7-9d85-a6dac86ce588 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:22:14.623976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b019c5-0791-48ae-b94a-3d68a40e6384 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:22:14.624557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b019c5-0791-48ae-b94a-3d68a40e6384 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:14.624833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b019c5-0791-48ae-b94a-3d68a40e6384 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:14.625113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b019c5-0791-48ae-b94a-3d68a40e6384 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:14.633137 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b019c5-0791-48ae-b94a-3d68a40e6384 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:22:14.634738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b019c5-0791-48ae-b94a-3d68a40e6384 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:22:14.644218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 758/1521] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:22:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:22:22.666053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639c3bc-b943-4ae0-b16a-114438079737 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:22.666608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639c3bc-b943-4ae0-b16a-114438079737 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:22.666608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639c3bc-b943-4ae0-b16a-114438079737 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:22.667961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 17:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:22:22.674535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947d7e0-c4f9-4035-a0b7-272068637ad9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:22.674991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947d7e0-c4f9-4035-a0b7-272068637ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:22.675101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947d7e0-c4f9-4035-a0b7-272068637ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:22.694210 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0947d7e0-c4f9-4035-a0b7-272068637ad9 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:22:22.695408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0947d7e0-c4f9-4035-a0b7-272068637ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:22.703225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 17:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:22:22.858383 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639c3bc-b943-4ae0-b16a-114438079737 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:22:22.872652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639c3bc-b943-4ae0-b16a-114438079737 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:22:22.878286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5077d-5518-4b28-9214-8a7a79db796a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:22:22.878877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5077d-5518-4b28-9214-8a7a79db796a service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:22.879170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5077d-5518-4b28-9214-8a7a79db796a service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:22.879418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5077d-5518-4b28-9214-8a7a79db796a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:22.887902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5077d-5518-4b28-9214-8a7a79db796a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:22:22.889455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5077d-5518-4b28-9214-8a7a79db796a service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:22:22.898592 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 765/1524] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:22:23.992416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f152c-0832-4ed8-ade0-77dc562c80b5 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:23.992968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f152c-0832-4ed8-ade0-77dc562c80b5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:23.992968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f152c-0832-4ed8-ade0-77dc562c80b5 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:23.994345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 27 17:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:22:24.001857 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f6309-3891-44a1-a142-1ca53f97541a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:24.002352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f6309-3891-44a1-a142-1ca53f97541a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:24.002459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f6309-3891-44a1-a142-1ca53f97541a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:24.021374 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b91f6309-3891-44a1-a142-1ca53f97541a None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:22:24.022610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b91f6309-3891-44a1-a142-1ca53f97541a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:24.030906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 27 17:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:22:24.184375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f152c-0832-4ed8-ade0-77dc562c80b5 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:22:24.198719 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f152c-0832-4ed8-ade0-77dc562c80b5 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:22:24.204098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6372e17-28e4-47e4-9155-ba46e3f1a2a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:22:24.204683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6372e17-28e4-47e4-9155-ba46e3f1a2a8 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:24.204956 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6372e17-28e4-47e4-9155-ba46e3f1a2a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:24.205225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6372e17-28e4-47e4-9155-ba46e3f1a2a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:24.213011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6372e17-28e4-47e4-9155-ba46e3f1a2a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:22:24.214616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6372e17-28e4-47e4-9155-ba46e3f1a2a8 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:22:24.224342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 761/1527] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:22:28.240189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b5b75-bec9-4d60-b098-98e592980356 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:28.240756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b5b75-bec9-4d60-b098-98e592980356 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:28.240756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b5b75-bec9-4d60-b098-98e592980356 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:28.242099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:22:28.248026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4164b3-2eea-45fa-91a2-057b9412bce2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:28.248445 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4164b3-2eea-45fa-91a2-057b9412bce2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:28.248554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4164b3-2eea-45fa-91a2-057b9412bce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:28.267650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-be4164b3-2eea-45fa-91a2-057b9412bce2 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:22:28.268873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-be4164b3-2eea-45fa-91a2-057b9412bce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:28.276701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:22:28.567368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b5b75-bec9-4d60-b098-98e592980356 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:22:28.569107 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-da3b5b75-bec9-4d60-b098-98e592980356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:28.646017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b5b75-bec9-4d60-b098-98e592980356 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:22:28.649508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed9c44-9669-4180-9d0b-b092366b6109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:22:28.650118 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed9c44-9669-4180-9d0b-b092366b6109 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:28.650395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed9c44-9669-4180-9d0b-b092366b6109 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:28.650660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed9c44-9669-4180-9d0b-b092366b6109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:28.659690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed9c44-9669-4180-9d0b-b092366b6109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:22:28.661094 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed9c44-9669-4180-9d0b-b092366b6109 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:22:28.671031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 768/1530] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:22:30.802859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4fa5d-5069-4a6c-9206-08c033d4fc36 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:30.803405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4fa5d-5069-4a6c-9206-08c033d4fc36 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:30.803405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4fa5d-5069-4a6c-9206-08c033d4fc36 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:30.804738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:22:30.810589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589ecac-489a-4ade-be3f-6f9e439a27d5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:30.811017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589ecac-489a-4ade-be3f-6f9e439a27d5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:30.811160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589ecac-489a-4ade-be3f-6f9e439a27d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:30.830811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5589ecac-489a-4ade-be3f-6f9e439a27d5 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:22:30.832007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5589ecac-489a-4ade-be3f-6f9e439a27d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:30.839745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:22:31.125302 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4fa5d-5069-4a6c-9206-08c033d4fc36 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:22:31.135782 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4fa5d-5069-4a6c-9206-08c033d4fc36 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:22:31.139344 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a3efb-a6c7-4fc2-a9f6-ac70403691af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:22:31.139929 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a3efb-a6c7-4fc2-a9f6-ac70403691af service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:31.140205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a3efb-a6c7-4fc2-a9f6-ac70403691af service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:31.140475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a3efb-a6c7-4fc2-a9f6-ac70403691af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:31.149070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a3efb-a6c7-4fc2-a9f6-ac70403691af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:22:31.150538 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a3efb-a6c7-4fc2-a9f6-ac70403691af service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:22:31.159686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 764/1533] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:22:35.091113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d550d77-a559-4f3f-86a5-3d951284d497 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:35.091684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d550d77-a559-4f3f-86a5-3d951284d497 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:35.091684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d550d77-a559-4f3f-86a5-3d951284d497 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:35.092977 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:22:35.098616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc296c2d-8b70-4dcd-8b40-ce39e40af184 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:35.099034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc296c2d-8b70-4dcd-8b40-ce39e40af184 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:35.099185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc296c2d-8b70-4dcd-8b40-ce39e40af184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:35.107267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=74 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:22:35.122622 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-bc296c2d-8b70-4dcd-8b40-ce39e40af184 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:22:35.123803 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-bc296c2d-8b70-4dcd-8b40-ce39e40af184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:35.132127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:22:35.435807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d550d77-a559-4f3f-86a5-3d951284d497 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:22:35.437492 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1d550d77-a559-4f3f-86a5-3d951284d497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:35.483327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:22:35.513765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d550d77-a559-4f3f-86a5-3d951284d497 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:22:35.517298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f5073-d304-4491-b9ac-1a7cf632215c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:22:35.517886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f5073-d304-4491-b9ac-1a7cf632215c service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:35.518193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f5073-d304-4491-b9ac-1a7cf632215c service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:35.518462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f5073-d304-4491-b9ac-1a7cf632215c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:35.527152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f5073-d304-4491-b9ac-1a7cf632215c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:22:35.528568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f5073-d304-4491-b9ac-1a7cf632215c service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:22:35.538228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 771/1536] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:22:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:22:37.660690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f6dce-b97a-48a8-8acc-205fcecbc889 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:37.661299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f6dce-b97a-48a8-8acc-205fcecbc889 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:37.661299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f6dce-b97a-48a8-8acc-205fcecbc889 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:37.662644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 27 17:22:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:22:37.667897 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc24ba-7361-42a7-960d-8efae5e79904 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:37.668349 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc24ba-7361-42a7-960d-8efae5e79904 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:37.668460 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc24ba-7361-42a7-960d-8efae5e79904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:37.687954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-19dc24ba-7361-42a7-960d-8efae5e79904 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:22:37.689151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-19dc24ba-7361-42a7-960d-8efae5e79904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:37.696924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 27 17:22:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:22:38.008012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f6dce-b97a-48a8-8acc-205fcecbc889 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:22:38.009674 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d18f6dce-b97a-48a8-8acc-205fcecbc889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:22:38.023250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f6dce-b97a-48a8-8acc-205fcecbc889 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:22:38.026751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc2b03-eb43-414a-9c00-ed71b60b1c0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:22:38.027335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc2b03-eb43-414a-9c00-ed71b60b1c0a service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:22:38.027611 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc2b03-eb43-414a-9c00-ed71b60b1c0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:22:38.027879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc2b03-eb43-414a-9c00-ed71b60b1c0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:22:38.037134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc2b03-eb43-414a-9c00-ed71b60b1c0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:22:38.038594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc2b03-eb43-414a-9c00-ed71b60b1c0a service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:22:38.047746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 767/1539] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:22:47.682677 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=18 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:22:48.043757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=8 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:23:59.831119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13eefe-c4de-4a47-8f0c-19331bbf1ad8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:23:59.831802 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13eefe-c4de-4a47-8f0c-19331bbf1ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:23:59.831802 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13eefe-c4de-4a47-8f0c-19331bbf1ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:23:59.856061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0c13eefe-c4de-4a47-8f0c-19331bbf1ad8 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:23:59.857837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0c13eefe-c4de-4a47-8f0c-19331bbf1ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:23:59.872893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 773/1540] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:23:59.881770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d4032-f7d9-4e63-9b6d-39a86fc76097 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:23:59.882402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d4032-f7d9-4e63-9b6d-39a86fc76097 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:23:59.882402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d4032-f7d9-4e63-9b6d-39a86fc76097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:23:59.911745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-989d4032-f7d9-4e63-9b6d-39a86fc76097 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:23:59.912967 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-989d4032-f7d9-4e63-9b6d-39a86fc76097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:23:59.926721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 768/1541] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:23:59.931279 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13eefe-c4de-4a47-8f0c-19331bbf1ad8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:23:59.945127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c13eefe-c4de-4a47-8f0c-19331bbf1ad8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:23:59.954412 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e427829-5bec-4f27-b57e-23fbf45a053b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:23:59.955048 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427829-5bec-4f27-b57e-23fbf45a053b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:23:59.955328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427829-5bec-4f27-b57e-23fbf45a053b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:23:59.955600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e427829-5bec-4f27-b57e-23fbf45a053b admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:23:59.956277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e427829-5bec-4f27-b57e-23fbf45a053b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:23:59.956579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e427829-5bec-4f27-b57e-23fbf45a053b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:23:59.958242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e427829-5bec-4f27-b57e-23fbf45a053b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:23:59.973897 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 774/1542] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:23:59.982567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d4032-f7d9-4e63-9b6d-39a86fc76097 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:23:59.995431 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d4032-f7d9-4e63-9b6d-39a86fc76097 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:00.007095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834ab8a-5293-44ef-b9c0-768ea81e1cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:00.008237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834ab8a-5293-44ef-b9c0-768ea81e1cbe admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.008698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834ab8a-5293-44ef-b9c0-768ea81e1cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.009013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834ab8a-5293-44ef-b9c0-768ea81e1cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.009813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834ab8a-5293-44ef-b9c0-768ea81e1cbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:00.012048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834ab8a-5293-44ef-b9c0-768ea81e1cbe admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:00.065342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 769/1543] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:23:59 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:00.069725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e427829-5bec-4f27-b57e-23fbf45a053b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:00.081854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e427829-5bec-4f27-b57e-23fbf45a053b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:00.156980 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01bfbe-1969-47c8-b2bb-c5e597c96f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:00.157603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01bfbe-1969-47c8-b2bb-c5e597c96f93 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.158087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01bfbe-1969-47c8-b2bb-c5e597c96f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.158372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01bfbe-1969-47c8-b2bb-c5e597c96f93 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.159110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01bfbe-1969-47c8-b2bb-c5e597c96f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:00.161303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c01bfbe-1969-47c8-b2bb-c5e597c96f93 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:00.164075 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e718e7d-9136-4cb8-8a04-4651ad5f1604 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.164374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e718e7d-9136-4cb8-8a04-4651ad5f1604 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.164652 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e718e7d-9136-4cb8-8a04-4651ad5f1604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.200202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 775/1544] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:00.201975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b50c56-08ac-45ce-a3c8-28d89146bd51 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.202288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b50c56-08ac-45ce-a3c8-28d89146bd51 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.202565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b50c56-08ac-45ce-a3c8-28d89146bd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.211959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1e718e7d-9136-4cb8-8a04-4651ad5f1604 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.213221 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1e718e7d-9136-4cb8-8a04-4651ad5f1604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.232182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 770/1545] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.233459 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e718e7d-9136-4cb8-8a04-4651ad5f1604 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:00.256945 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e718e7d-9136-4cb8-8a04-4651ad5f1604 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:00.271717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02903b36-bd28-4d6d-aabc-63b5189b3d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:00.272406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02903b36-bd28-4d6d-aabc-63b5189b3d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.272688 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02903b36-bd28-4d6d-aabc-63b5189b3d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.272959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02903b36-bd28-4d6d-aabc-63b5189b3d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.273859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02903b36-bd28-4d6d-aabc-63b5189b3d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:00.275518 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02903b36-bd28-4d6d-aabc-63b5189b3d44 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:00.288007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 771/1546] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:00.289661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeb2ce-c22c-415d-8d8f-aac507bc5675 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.289942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeb2ce-c22c-415d-8d8f-aac507bc5675 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.293291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aeb2ce-c22c-415d-8d8f-aac507bc5675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.297774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-14b50c56-08ac-45ce-a3c8-28d89146bd51 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.299082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-14b50c56-08ac-45ce-a3c8-28d89146bd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.324274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-26aeb2ce-c22c-415d-8d8f-aac507bc5675 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.324274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-26aeb2ce-c22c-415d-8d8f-aac507bc5675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.330676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 776/1547] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.332207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd638e-76e7-4409-946b-f00be13c101e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.333848 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd638e-76e7-4409-946b-f00be13c101e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.338933 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd638e-76e7-4409-946b-f00be13c101e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.343863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 772/1548] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.345352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aeb2ce-c22c-415d-8d8f-aac507bc5675 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:00.359946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aeb2ce-c22c-415d-8d8f-aac507bc5675 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:00.371080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6bda3-7ab0-4009-9b91-4747651b26f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:00.371704 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6bda3-7ab0-4009-9b91-4747651b26f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.371978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6bda3-7ab0-4009-9b91-4747651b26f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.372281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6bda3-7ab0-4009-9b91-4747651b26f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.373016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6bda3-7ab0-4009-9b91-4747651b26f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:00.377560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6bda3-7ab0-4009-9b91-4747651b26f9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:00.383869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0cbd638e-76e7-4409-946b-f00be13c101e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.385104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0cbd638e-76e7-4409-946b-f00be13c101e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.399508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 773/1549] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:00.400788 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6bda3-7ab0-4009-9b91-4747651b26f9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:00.405973 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 777/1550] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.407308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c82fd-9135-46ce-9471-d718ea23bebb None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.411126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c82fd-9135-46ce-9471-d718ea23bebb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.411126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c82fd-9135-46ce-9471-d718ea23bebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.419830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6bda3-7ab0-4009-9b91-4747651b26f9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:00.426350 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3049509-75bd-4361-a901-9286169c0c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:00.427044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3049509-75bd-4361-a901-9286169c0c23 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.427396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3049509-75bd-4361-a901-9286169c0c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.427755 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3049509-75bd-4361-a901-9286169c0c23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.428766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3049509-75bd-4361-a901-9286169c0c23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:00.429146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3049509-75bd-4361-a901-9286169c0c23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:00.430340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-c3049509-75bd-4361-a901-9286169c0c23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:24:00.430340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 27 17:24:00.430340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: Sep 27 17:24:00.431547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3049509-75bd-4361-a901-9286169c0c23 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:00.434564 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bd9c82fd-9135-46ce-9471-d718ea23bebb None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.435856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bd9c82fd-9135-46ce-9471-d718ea23bebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.444384 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 778/1551] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.445352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 774/1552] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:00.445479 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96119888-1265-486a-9caa-a3454e55eee3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.445708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96119888-1265-486a-9caa-a3454e55eee3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.445944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96119888-1265-486a-9caa-a3454e55eee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.450404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7b478-673c-4cd8-b240-d677c797b8f0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.450404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7b478-673c-4cd8-b240-d677c797b8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.450404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f7b478-673c-4cd8-b240-d677c797b8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.469591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-33f7b478-673c-4cd8-b240-d677c797b8f0 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.470333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-96119888-1265-486a-9caa-a3454e55eee3 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.470900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-33f7b478-673c-4cd8-b240-d677c797b8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.471528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-96119888-1265-486a-9caa-a3454e55eee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.481488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 775/1553] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.481488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7b478-673c-4cd8-b240-d677c797b8f0 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:00.488209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 779/1554] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.489616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8599d90-869b-4c9f-a873-4ec6e04e48dc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.489901 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8599d90-869b-4c9f-a873-4ec6e04e48dc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.490216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8599d90-869b-4c9f-a873-4ec6e04e48dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.491475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f7b478-673c-4cd8-b240-d677c797b8f0 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:00.496582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2426ec-7350-4519-a072-aeeb41158031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:00.498549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2426ec-7350-4519-a072-aeeb41158031 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.498831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2426ec-7350-4519-a072-aeeb41158031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.499104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2426ec-7350-4519-a072-aeeb41158031 admin admin] PATH_INFO: `/v3/projects/0b7fcad3ea934d18b26586a80661b549/users/766279c94408482a8438cb44bebadbc5/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.510377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c8599d90-869b-4c9f-a873-4ec6e04e48dc None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.511585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c8599d90-869b-4c9f-a873-4ec6e04e48dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.514313 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2426ec-7350-4519-a072-aeeb41158031 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7fcad3ea934d18b26586a80661b549, user_id=766279c94408482a8438cb44bebadbc5, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:00.516963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2426ec-7350-4519-a072-aeeb41158031 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:00.520101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 780/1555] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.521481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985a0da-d160-4226-a58b-5c433006228b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.521924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985a0da-d160-4226-a58b-5c433006228b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.522281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985a0da-d160-4226-a58b-5c433006228b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.532518 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 776/1556] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:00 2022] PUT /identity/v3/projects/0b7fcad3ea934d18b26586a80661b549/users/766279c94408482a8438cb44bebadbc5/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:00.532518 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2426ec-7350-4519-a072-aeeb41158031 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:00.543312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1985a0da-d160-4226-a58b-5c433006228b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.545430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1985a0da-d160-4226-a58b-5c433006228b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.601112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2426ec-7350-4519-a072-aeeb41158031 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:00.605973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5362f-ccd8-4382-860b-f9d92c5f56dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:00.606599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5362f-ccd8-4382-860b-f9d92c5f56dc admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.607395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5362f-ccd8-4382-860b-f9d92c5f56dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.607667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5362f-ccd8-4382-860b-f9d92c5f56dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.608467 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5362f-ccd8-4382-860b-f9d92c5f56dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:00.608771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5362f-ccd8-4382-860b-f9d92c5f56dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:00.610158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 781/1557] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.610446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5362f-ccd8-4382-860b-f9d92c5f56dc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:00.611016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985a0da-d160-4226-a58b-5c433006228b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:00.620757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985a0da-d160-4226-a58b-5c433006228b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:00.624804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 777/1558] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:00.626162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd432dd-9b53-49a4-9952-806a8d6398eb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.626443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd432dd-9b53-49a4-9952-806a8d6398eb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.626711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd432dd-9b53-49a4-9952-806a8d6398eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.632422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba458812-5c8b-4b4e-b89f-824dfda35d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:00.633021 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458812-5c8b-4b4e-b89f-824dfda35d89 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.633299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458812-5c8b-4b4e-b89f-824dfda35d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.633607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba458812-5c8b-4b4e-b89f-824dfda35d89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.634474 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba458812-5c8b-4b4e-b89f-824dfda35d89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:00.636416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-ba458812-5c8b-4b4e-b89f-824dfda35d89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:24:00.636416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 27 17:24:00.636416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: Sep 27 17:24:00.637624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba458812-5c8b-4b4e-b89f-824dfda35d89 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:00.646055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0dd432dd-9b53-49a4-9952-806a8d6398eb None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.647271 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0dd432dd-9b53-49a4-9952-806a8d6398eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.655763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 778/1559] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.657032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbefc8-c7fb-42ae-b339-694a5a4ca9f0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.657295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbefc8-c7fb-42ae-b339-694a5a4ca9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.657567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbefc8-c7fb-42ae-b339-694a5a4ca9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.675411 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 782/1560] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:00.676747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cea8a-e071-46d0-9a6f-ae24aecd166f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.677090 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cea8a-e071-46d0-9a6f-ae24aecd166f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.677419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15cea8a-e071-46d0-9a6f-ae24aecd166f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.679593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5bfbefc8-c7fb-42ae-b339-694a5a4ca9f0 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.680855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5bfbefc8-c7fb-42ae-b339-694a5a4ca9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.697383 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d15cea8a-e071-46d0-9a6f-ae24aecd166f None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.698632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d15cea8a-e071-46d0-9a6f-ae24aecd166f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.708676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:00.710287 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:00.745489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 779/1561] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.746375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbefc8-c7fb-42ae-b339-694a5a4ca9f0 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:00.755371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbefc8-c7fb-42ae-b339-694a5a4ca9f0 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:00.760198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b716837-4325-4348-afd9-c275d03bf57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:00.760539 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b716837-4325-4348-afd9-c275d03bf57e admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.760810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b716837-4325-4348-afd9-c275d03bf57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.761086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b716837-4325-4348-afd9-c275d03bf57e admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.761730 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b716837-4325-4348-afd9-c275d03bf57e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:00.762029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b716837-4325-4348-afd9-c275d03bf57e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:00.763690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b716837-4325-4348-afd9-c275d03bf57e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:00.771808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 783/1562] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.773167 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b50c7-d779-4730-8a76-b60b8fda389b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.773456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b50c7-d779-4730-8a76-b60b8fda389b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.773715 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b50c7-d779-4730-8a76-b60b8fda389b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.775621 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 780/1563] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:00.776709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed90ee-c292-4809-8a82-1be5d67f8f21 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.776991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed90ee-c292-4809-8a82-1be5d67f8f21 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.777263 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed90ee-c292-4809-8a82-1be5d67f8f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.803884 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ceed90ee-c292-4809-8a82-1be5d67f8f21 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.805100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ceed90ee-c292-4809-8a82-1be5d67f8f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.807844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f31b50c7-d779-4730-8a76-b60b8fda389b None None] MFA Rules not processed for user `766279c94408482a8438cb44bebadbc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.809092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f31b50c7-d779-4730-8a76-b60b8fda389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.813797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 781/1564] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.814387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed90ee-c292-4809-8a82-1be5d67f8f21 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:00.823832 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed90ee-c292-4809-8a82-1be5d67f8f21 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:00.828933 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3067e7-dc5e-4de0-99bc-a0601cae7649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:00.829554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3067e7-dc5e-4de0-99bc-a0601cae7649 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.829867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3067e7-dc5e-4de0-99bc-a0601cae7649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.830198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3067e7-dc5e-4de0-99bc-a0601cae7649 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.830936 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3067e7-dc5e-4de0-99bc-a0601cae7649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:00.832906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3067e7-dc5e-4de0-99bc-a0601cae7649 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:00.862261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 782/1565] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:00.863551 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dff0f-a11e-4816-bd7c-75ab52e89f83 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.863843 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dff0f-a11e-4816-bd7c-75ab52e89f83 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.864104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638dff0f-a11e-4816-bd7c-75ab52e89f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.883530 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-638dff0f-a11e-4816-bd7c-75ab52e89f83 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:00.884732 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-638dff0f-a11e-4816-bd7c-75ab52e89f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:00.887765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 784/1566] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.893211 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b50c7-d779-4730-8a76-b60b8fda389b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:00.903013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b50c7-d779-4730-8a76-b60b8fda389b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:00.949279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 783/1567] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:00.951124 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dff0f-a11e-4816-bd7c-75ab52e89f83 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:00.960452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638dff0f-a11e-4816-bd7c-75ab52e89f83 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:00.965352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb922d0-935e-499b-a361-e7de24317af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:00.965506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0d89a-598f-4850-8e12-0c502423bc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:00.965991 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0d89a-598f-4850-8e12-0c502423bc13 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.966662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb922d0-935e-499b-a361-e7de24317af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:00.966790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0d89a-598f-4850-8e12-0c502423bc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.966938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb922d0-935e-499b-a361-e7de24317af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:00.967064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0d89a-598f-4850-8e12-0c502423bc13 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.967208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb922d0-935e-499b-a361-e7de24317af6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:00.967710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0d89a-598f-4850-8e12-0c502423bc13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:00.967839 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb922d0-935e-499b-a361-e7de24317af6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:00.968168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb922d0-935e-499b-a361-e7de24317af6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:00.969645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0d89a-598f-4850-8e12-0c502423bc13 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:00.969752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb922d0-935e-499b-a361-e7de24317af6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:00.990138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 784/1568] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:00.990138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb922d0-935e-499b-a361-e7de24317af6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:00.993191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 785/1569] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:00.994197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0d89a-598f-4850-8e12-0c502423bc13 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.000970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb922d0-935e-499b-a361-e7de24317af6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.004766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0d89a-598f-4850-8e12-0c502423bc13 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.006644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1ab1c-aa55-425b-9d14-863814d0d63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.007362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1ab1c-aa55-425b-9d14-863814d0d63d admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.007639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1ab1c-aa55-425b-9d14-863814d0d63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.008051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1ab1c-aa55-425b-9d14-863814d0d63d admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.008933 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1ab1c-aa55-425b-9d14-863814d0d63d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:01.009235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1ab1c-aa55-425b-9d14-863814d0d63d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.009911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae11f04-0a45-45b5-adbb-fc0eb438a01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.010582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae11f04-0a45-45b5-adbb-fc0eb438a01f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.010898 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae11f04-0a45-45b5-adbb-fc0eb438a01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.011008 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1ab1c-aa55-425b-9d14-863814d0d63d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.011192 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae11f04-0a45-45b5-adbb-fc0eb438a01f admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.011879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae11f04-0a45-45b5-adbb-fc0eb438a01f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.014137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae11f04-0a45-45b5-adbb-fc0eb438a01f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.023093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 785/1570] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:01.030807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1ab1c-aa55-425b-9d14-863814d0d63d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.035026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1ab1c-aa55-425b-9d14-863814d0d63d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.040182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa96124-ed07-43fe-878b-17b24006ef00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.040789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa96124-ed07-43fe-878b-17b24006ef00 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.041061 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa96124-ed07-43fe-878b-17b24006ef00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.041462 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa96124-ed07-43fe-878b-17b24006ef00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.042155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa96124-ed07-43fe-878b-17b24006ef00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:01.042453 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa96124-ed07-43fe-878b-17b24006ef00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.043287 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 786/1571] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:00 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:01.044159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae11f04-0a45-45b5-adbb-fc0eb438a01f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.044486 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa96124-ed07-43fe-878b-17b24006ef00 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.063289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 786/1572] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:01.063289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c9fc7-7419-47a0-815d-2fdf11ebde70 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.063289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c9fc7-7419-47a0-815d-2fdf11ebde70 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.063289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c9fc7-7419-47a0-815d-2fdf11ebde70 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.063289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 787/1573] 10.222.0.55 () {54 vars in 961 bytes} [Tue Sep 27 17:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:24:01.063289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c9fc7-7419-47a0-815d-2fdf11ebde70 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.065117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:01.114490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae11f04-0a45-45b5-adbb-fc0eb438a01f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.114870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:01.119949 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf13589-fef6-41fa-aa5d-a2e27bc5eeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.120713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf13589-fef6-41fa-aa5d-a2e27bc5eeed admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.121038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf13589-fef6-41fa-aa5d-a2e27bc5eeed admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.121346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf13589-fef6-41fa-aa5d-a2e27bc5eeed admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.121980 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf13589-fef6-41fa-aa5d-a2e27bc5eeed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.124053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf13589-fef6-41fa-aa5d-a2e27bc5eeed admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.128477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c9fc7-7419-47a0-815d-2fdf11ebde70 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.132956 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6515b-87be-4eb1-bac0-4c24d018151c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.133552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6515b-87be-4eb1-bac0-4c24d018151c admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.133829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6515b-87be-4eb1-bac0-4c24d018151c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.134123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6515b-87be-4eb1-bac0-4c24d018151c admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.134771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6515b-87be-4eb1-bac0-4c24d018151c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:01.135070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6515b-87be-4eb1-bac0-4c24d018151c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.136639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6515b-87be-4eb1-bac0-4c24d018151c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.148157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 787/1574] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:01 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:01.148996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf13589-fef6-41fa-aa5d-a2e27bc5eeed admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.156897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 788/1575] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:01.156897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6515b-87be-4eb1-bac0-4c24d018151c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.165891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6515b-87be-4eb1-bac0-4c24d018151c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.166763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf13589-fef6-41fa-aa5d-a2e27bc5eeed admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.171130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1131c-52c7-4859-91c3-b1bab604549f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.171710 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1131c-52c7-4859-91c3-b1bab604549f admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.171985 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1131c-52c7-4859-91c3-b1bab604549f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.172247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd22bc-158e-4d23-8bc6-73846f892e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.172375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1131c-52c7-4859-91c3-b1bab604549f admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.172876 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1131c-52c7-4859-91c3-b1bab604549f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.173023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd22bc-158e-4d23-8bc6-73846f892e5f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.173189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd22bc-158e-4d23-8bc6-73846f892e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.173493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd22bc-158e-4d23-8bc6-73846f892e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.174188 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd22bc-158e-4d23-8bc6-73846f892e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.174491 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1131c-52c7-4859-91c3-b1bab604549f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.176272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd22bc-158e-4d23-8bc6-73846f892e5f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.184420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 789/1576] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:01.185755 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1131c-52c7-4859-91c3-b1bab604549f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.195235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1131c-52c7-4859-91c3-b1bab604549f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.199906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bfdbd-722e-4660-a04b-d980fbaa7a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.200519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bfdbd-722e-4660-a04b-d980fbaa7a11 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.200815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bfdbd-722e-4660-a04b-d980fbaa7a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.201061 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578bfdbd-722e-4660-a04b-d980fbaa7a11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.201675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578bfdbd-722e-4660-a04b-d980fbaa7a11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.203595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578bfdbd-722e-4660-a04b-d980fbaa7a11 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.208298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 788/1577] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:01 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:01.208298 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd22bc-158e-4d23-8bc6-73846f892e5f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.236851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 790/1578] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:01 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:01.236851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bfdbd-722e-4660-a04b-d980fbaa7a11 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.276325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd22bc-158e-4d23-8bc6-73846f892e5f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.295999 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578bfdbd-722e-4660-a04b-d980fbaa7a11 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.300702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999febea-1144-489b-8261-7e9d086db87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.301374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999febea-1144-489b-8261-7e9d086db87f admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.301656 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999febea-1144-489b-8261-7e9d086db87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.301934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999febea-1144-489b-8261-7e9d086db87f admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.302595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999febea-1144-489b-8261-7e9d086db87f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.304534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999febea-1144-489b-8261-7e9d086db87f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.333619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 791/1579] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:01 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:01.334933 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83cb31-a008-4280-b629-258a0c14f9cd None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.335224 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83cb31-a008-4280-b629-258a0c14f9cd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.335494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83cb31-a008-4280-b629-258a0c14f9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.341034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dccab4-e3e4-4b18-9680-6ad383c3210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.341698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dccab4-e3e4-4b18-9680-6ad383c3210b admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.341978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dccab4-e3e4-4b18-9680-6ad383c3210b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.342322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dccab4-e3e4-4b18-9680-6ad383c3210b admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.342974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dccab4-e3e4-4b18-9680-6ad383c3210b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.344885 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dccab4-e3e4-4b18-9680-6ad383c3210b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.367533 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 789/1580] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:01 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:01.368276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dccab4-e3e4-4b18-9680-6ad383c3210b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.377097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0e83cb31-a008-4280-b629-258a0c14f9cd None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:01.378426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0e83cb31-a008-4280-b629-258a0c14f9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:01.380299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dccab4-e3e4-4b18-9680-6ad383c3210b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.393109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:01.439339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ca064-2b15-484e-9e46-f2a75b7b8afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.439990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ca064-2b15-484e-9e46-f2a75b7b8afb admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.440263 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ca064-2b15-484e-9e46-f2a75b7b8afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.440532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ca064-2b15-484e-9e46-f2a75b7b8afb admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.441165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ca064-2b15-484e-9e46-f2a75b7b8afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.443061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ca064-2b15-484e-9e46-f2a75b7b8afb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.446847 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 792/1581] 10.222.0.55 () {58 vars in 1058 bytes} [Tue Sep 27 17:24:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:01.447536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83cb31-a008-4280-b629-258a0c14f9cd None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.453349 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 790/1582] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:01.454606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ca064-2b15-484e-9e46-f2a75b7b8afb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.460170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83cb31-a008-4280-b629-258a0c14f9cd None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.465055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0d89a-e5b3-41f1-ae14-a9fadcb34fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.465245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ca064-2b15-484e-9e46-f2a75b7b8afb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.465656 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0d89a-e5b3-41f1-ae14-a9fadcb34fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.465936 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0d89a-e5b3-41f1-ae14-a9fadcb34fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.466840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd0d89a-e5b3-41f1-ae14-a9fadcb34fe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.467421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0d89a-e5b3-41f1-ae14-a9fadcb34fe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.469311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd0d89a-e5b3-41f1-ae14-a9fadcb34fe9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.470430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a175015-1ed5-4c0c-8b93-2ba452c9a481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.471014 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a175015-1ed5-4c0c-8b93-2ba452c9a481 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.471290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a175015-1ed5-4c0c-8b93-2ba452c9a481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.471550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a175015-1ed5-4c0c-8b93-2ba452c9a481 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.472176 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a175015-1ed5-4c0c-8b93-2ba452c9a481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.473760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a175015-1ed5-4c0c-8b93-2ba452c9a481 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.487625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 791/1583] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:01.487625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a175015-1ed5-4c0c-8b93-2ba452c9a481 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.488060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:01.498416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a175015-1ed5-4c0c-8b93-2ba452c9a481 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.500283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 793/1584] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:01 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:01.500954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0d89a-e5b3-41f1-ae14-a9fadcb34fe9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.558665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4f9cc-1fea-46fe-a80b-9ab95a28f49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.559454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4f9cc-1fea-46fe-a80b-9ab95a28f49c admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.560340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4f9cc-1fea-46fe-a80b-9ab95a28f49c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.560719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4f9cc-1fea-46fe-a80b-9ab95a28f49c admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.561414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4f9cc-1fea-46fe-a80b-9ab95a28f49c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.563345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4f9cc-1fea-46fe-a80b-9ab95a28f49c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.564489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd0d89a-e5b3-41f1-ae14-a9fadcb34fe9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.569712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72a109-eeaf-4d64-8e02-d86bfea4e4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.570364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72a109-eeaf-4d64-8e02-d86bfea4e4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.570642 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72a109-eeaf-4d64-8e02-d86bfea4e4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.571009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72a109-eeaf-4d64-8e02-d86bfea4e4b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.571662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72a109-eeaf-4d64-8e02-d86bfea4e4b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.572740 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-9e72a109-eeaf-4d64-8e02-d86bfea4e4b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:24:01.572740 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 27 17:24:01.572740 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: Sep 27 17:24:01.573979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72a109-eeaf-4d64-8e02-d86bfea4e4b7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.585151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 792/1585] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:01 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:01.586064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4f9cc-1fea-46fe-a80b-9ab95a28f49c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.597800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4f9cc-1fea-46fe-a80b-9ab95a28f49c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.600340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 794/1586] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:01 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:01.601165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72a109-eeaf-4d64-8e02-d86bfea4e4b7 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.605303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559244f-f3b2-4867-9353-4325f0edfe6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.605980 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559244f-f3b2-4867-9353-4325f0edfe6f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.606311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559244f-f3b2-4867-9353-4325f0edfe6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.606587 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559244f-f3b2-4867-9353-4325f0edfe6f admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.607819 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559244f-f3b2-4867-9353-4325f0edfe6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.609800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559244f-f3b2-4867-9353-4325f0edfe6f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.612100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72a109-eeaf-4d64-8e02-d86bfea4e4b7 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.616721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1128a6-08b5-47f4-bc7d-c2a3d12bd9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.617300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1128a6-08b5-47f4-bc7d-c2a3d12bd9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.617642 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1128a6-08b5-47f4-bc7d-c2a3d12bd9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.617847 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1128a6-08b5-47f4-bc7d-c2a3d12bd9e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.618516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1128a6-08b5-47f4-bc7d-c2a3d12bd9e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.620102 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1128a6-08b5-47f4-bc7d-c2a3d12bd9e6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.630926 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 795/1587] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:01.631720 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1128a6-08b5-47f4-bc7d-c2a3d12bd9e6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.635238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 793/1588] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:01 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:01.636230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559244f-f3b2-4867-9353-4325f0edfe6f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.645798 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559244f-f3b2-4867-9353-4325f0edfe6f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.651050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf81c7-fe5f-481e-8adb-cb71941f23c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.651743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf81c7-fe5f-481e-8adb-cb71941f23c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.652117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf81c7-fe5f-481e-8adb-cb71941f23c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.652487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf81c7-fe5f-481e-8adb-cb71941f23c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.653220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf81c7-fe5f-481e-8adb-cb71941f23c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.655482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf81c7-fe5f-481e-8adb-cb71941f23c8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.666131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 794/1589] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:01.667070 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf81c7-fe5f-481e-8adb-cb71941f23c8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.677161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf81c7-fe5f-481e-8adb-cb71941f23c8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.681691 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc95518-6b89-425b-be7d-a445b6bc671d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.682348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc95518-6b89-425b-be7d-a445b6bc671d admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.682624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc95518-6b89-425b-be7d-a445b6bc671d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.682892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc95518-6b89-425b-be7d-a445b6bc671d admin admin] PATH_INFO: `/v3/projects/63e37b343da749f7a43804b2d1224288/users/c3bc72fa2ee34c268308bce70122babc/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.696824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc95518-6b89-425b-be7d-a445b6bc671d admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e37b343da749f7a43804b2d1224288, user_id=c3bc72fa2ee34c268308bce70122babc, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.697844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-9fc95518-6b89-425b-be7d-a445b6bc671d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:24:01.697844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 27 17:24:01.697844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: Sep 27 17:24:01.699796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc95518-6b89-425b-be7d-a445b6bc671d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.708288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1128a6-08b5-47f4-bc7d-c2a3d12bd9e6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.712968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 795/1590] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:01 2022] PUT /identity/v3/projects/63e37b343da749f7a43804b2d1224288/users/c3bc72fa2ee34c268308bce70122babc/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:01.720829 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc95518-6b89-425b-be7d-a445b6bc671d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.766597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:01.786849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc95518-6b89-425b-be7d-a445b6bc671d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.790928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569aa60-4033-4cff-aa18-8fa7b3acc5f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.791409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36ac85-966b-48da-a4f4-05b4ae9bdc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.791525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569aa60-4033-4cff-aa18-8fa7b3acc5f7 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.791796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569aa60-4033-4cff-aa18-8fa7b3acc5f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.792028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36ac85-966b-48da-a4f4-05b4ae9bdc92 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.792147 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569aa60-4033-4cff-aa18-8fa7b3acc5f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.792299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36ac85-966b-48da-a4f4-05b4ae9bdc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.792571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36ac85-966b-48da-a4f4-05b4ae9bdc92 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.793186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36ac85-966b-48da-a4f4-05b4ae9bdc92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.795699 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36ac85-966b-48da-a4f4-05b4ae9bdc92 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.798212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:01.817331 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 796/1591] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:01 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:01.818577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36ac85-966b-48da-a4f4-05b4ae9bdc92 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.828448 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36ac85-966b-48da-a4f4-05b4ae9bdc92 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.833142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194318f-56fa-4423-8a69-04cfcba84881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.833717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194318f-56fa-4423-8a69-04cfcba84881 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.833990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194318f-56fa-4423-8a69-04cfcba84881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.834300 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194318f-56fa-4423-8a69-04cfcba84881 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.834934 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194318f-56fa-4423-8a69-04cfcba84881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.836504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194318f-56fa-4423-8a69-04cfcba84881 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.849856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 797/1592] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:01.849856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194318f-56fa-4423-8a69-04cfcba84881 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.855503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569aa60-4033-4cff-aa18-8fa7b3acc5f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.857661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569aa60-4033-4cff-aa18-8fa7b3acc5f7 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.860884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194318f-56fa-4423-8a69-04cfcba84881 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.866759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d1714-f685-471a-bcd6-2e81105c1f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.866923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 796/1593] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:01.867387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1714-f685-471a-bcd6-2e81105c1f6d admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.867719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1714-f685-471a-bcd6-2e81105c1f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.867847 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569aa60-4033-4cff-aa18-8fa7b3acc5f7 service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.868014 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d1714-f685-471a-bcd6-2e81105c1f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.868689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d1714-f685-471a-bcd6-2e81105c1f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.870357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d1714-f685-471a-bcd6-2e81105c1f6d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.879152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569aa60-4033-4cff-aa18-8fa7b3acc5f7 service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.882213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 798/1594] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:01.884126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158ab8e-1355-4ab4-824a-3569aa391775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.885486 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158ab8e-1355-4ab4-824a-3569aa391775 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.886227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d1714-f685-471a-bcd6-2e81105c1f6d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.886361 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158ab8e-1355-4ab4-824a-3569aa391775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.886895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158ab8e-1355-4ab4-824a-3569aa391775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.887774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158ab8e-1355-4ab4-824a-3569aa391775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.890028 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5158ab8e-1355-4ab4-824a-3569aa391775 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.896389 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d1714-f685-471a-bcd6-2e81105c1f6d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.902691 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e507e-14d1-40b7-b4d8-b8f83d9f5355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.902857 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 797/1595] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:01.903187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e507e-14d1-40b7-b4d8-b8f83d9f5355 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.903473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e507e-14d1-40b7-b4d8-b8f83d9f5355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.903787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158ab8e-1355-4ab4-824a-3569aa391775 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:01.903908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e507e-14d1-40b7-b4d8-b8f83d9f5355 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.904421 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e507e-14d1-40b7-b4d8-b8f83d9f5355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.906106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e507e-14d1-40b7-b4d8-b8f83d9f5355 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.913582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158ab8e-1355-4ab4-824a-3569aa391775 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:01.918552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed712c5-f9ce-439b-84a3-da174e9c981d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:01.920158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 799/1596] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:01.920293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed712c5-f9ce-439b-84a3-da174e9c981d admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.920694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed712c5-f9ce-439b-84a3-da174e9c981d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.921078 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed712c5-f9ce-439b-84a3-da174e9c981d admin admin] PATH_INFO: `/v3/projects/9c16b9aaa5ac4e6dac613614c1a570c8/users/e48c146c0bba4583a92d157e2eed7f99/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.921317 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66d85a-6822-4c59-9bc0-f7143c77371e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:01.921627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66d85a-6822-4c59-9bc0-f7143c77371e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:01.921879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66d85a-6822-4c59-9bc0-f7143c77371e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:01.939495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed712c5-f9ce-439b-84a3-da174e9c981d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c16b9aaa5ac4e6dac613614c1a570c8, user_id=e48c146c0bba4583a92d157e2eed7f99, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:01.942592 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed712c5-f9ce-439b-84a3-da174e9c981d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:01.951289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6a66d85a-6822-4c59-9bc0-f7143c77371e None None] MFA Rules not processed for user `c3bc72fa2ee34c268308bce70122babc`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:01.952498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6a66d85a-6822-4c59-9bc0-f7143c77371e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:01.963264 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 798/1597] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:01 2022] PUT /identity/v3/projects/9c16b9aaa5ac4e6dac613614c1a570c8/users/e48c146c0bba4583a92d157e2eed7f99/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:01.963264 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed712c5-f9ce-439b-84a3-da174e9c981d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.023032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 800/1598] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:02.024097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66d85a-6822-4c59-9bc0-f7143c77371e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.031413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed712c5-f9ce-439b-84a3-da174e9c981d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.035457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66d85a-6822-4c59-9bc0-f7143c77371e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.036018 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896872b-f27d-4e1b-b11c-4fe57e7f6fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.036663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896872b-f27d-4e1b-b11c-4fe57e7f6fad admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.036968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896872b-f27d-4e1b-b11c-4fe57e7f6fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.037275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896872b-f27d-4e1b-b11c-4fe57e7f6fad admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.038016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896872b-f27d-4e1b-b11c-4fe57e7f6fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.039662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896872b-f27d-4e1b-b11c-4fe57e7f6fad admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.040807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37bc65-e4da-47a6-b58d-1802357d7547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.041365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37bc65-e4da-47a6-b58d-1802357d7547 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.041641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37bc65-e4da-47a6-b58d-1802357d7547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.042141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37bc65-e4da-47a6-b58d-1802357d7547 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.042560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37bc65-e4da-47a6-b58d-1802357d7547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.044270 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37bc65-e4da-47a6-b58d-1802357d7547 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.051018 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 799/1599] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:02.054758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896872b-f27d-4e1b-b11c-4fe57e7f6fad admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.058352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 801/1600] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:02.058352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37bc65-e4da-47a6-b58d-1802357d7547 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.067983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37bc65-e4da-47a6-b58d-1802357d7547 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.068231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896872b-f27d-4e1b-b11c-4fe57e7f6fad admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.072913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f93a1c-fa57-499c-b985-2c150a629a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.073185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502159d0-9e67-4b62-8402-e0c26d15f07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.073669 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f93a1c-fa57-499c-b985-2c150a629a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.073825 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502159d0-9e67-4b62-8402-e0c26d15f07b admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.074109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f93a1c-fa57-499c-b985-2c150a629a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.074245 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502159d0-9e67-4b62-8402-e0c26d15f07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.074477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502159d0-9e67-4b62-8402-e0c26d15f07b admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.074621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f93a1c-fa57-499c-b985-2c150a629a09 admin admin] PATH_INFO: `/v3/projects/94de95e80dbc4006b100d487e0f0921c/users/f6525c9ecf4741f5a377126110bf6085/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.075103 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502159d0-9e67-4b62-8402-e0c26d15f07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.077229 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502159d0-9e67-4b62-8402-e0c26d15f07b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.088289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 800/1601] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:02.092276 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502159d0-9e67-4b62-8402-e0c26d15f07b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.096614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f93a1c-fa57-499c-b985-2c150a629a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94de95e80dbc4006b100d487e0f0921c, user_id=f6525c9ecf4741f5a377126110bf6085, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.099285 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f93a1c-fa57-499c-b985-2c150a629a09 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.099974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502159d0-9e67-4b62-8402-e0c26d15f07b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.106542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b4ad1-8e1e-4eb2-a010-bc476739fc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.107169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b4ad1-8e1e-4eb2-a010-bc476739fc55 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.107445 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b4ad1-8e1e-4eb2-a010-bc476739fc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.107794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b4ad1-8e1e-4eb2-a010-bc476739fc55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.108484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b4ad1-8e1e-4eb2-a010-bc476739fc55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.110113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b4ad1-8e1e-4eb2-a010-bc476739fc55 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.118335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 802/1602] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:02 2022] PUT /identity/v3/projects/94de95e80dbc4006b100d487e0f0921c/users/f6525c9ecf4741f5a377126110bf6085/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:02.119288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d35cc6-87e1-4f8b-beb0-c08fbea78890 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.119572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d35cc6-87e1-4f8b-beb0-c08fbea78890 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.119968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d35cc6-87e1-4f8b-beb0-c08fbea78890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.124218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 801/1603] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:02.124218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b4ad1-8e1e-4eb2-a010-bc476739fc55 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.134424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b4ad1-8e1e-4eb2-a010-bc476739fc55 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.140912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-01d35cc6-87e1-4f8b-beb0-c08fbea78890 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:02.142239 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-01d35cc6-87e1-4f8b-beb0-c08fbea78890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:02.179455 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:02.187428 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:02.196981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680af31-80fa-4e15-aa8f-351844bbaf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.197643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680af31-80fa-4e15-aa8f-351844bbaf87 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.197924 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680af31-80fa-4e15-aa8f-351844bbaf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.198239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680af31-80fa-4e15-aa8f-351844bbaf87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.198872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680af31-80fa-4e15-aa8f-351844bbaf87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.201457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680af31-80fa-4e15-aa8f-351844bbaf87 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.203236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 803/1604] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:02.204197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d35cc6-87e1-4f8b-beb0-c08fbea78890 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.213614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d35cc6-87e1-4f8b-beb0-c08fbea78890 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.218377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bbf50-aea0-4779-a9db-1ce381d9a066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.218975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bbf50-aea0-4779-a9db-1ce381d9a066 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.219249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bbf50-aea0-4779-a9db-1ce381d9a066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.219515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bbf50-aea0-4779-a9db-1ce381d9a066 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.220148 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942bbf50-aea0-4779-a9db-1ce381d9a066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.221728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942bbf50-aea0-4779-a9db-1ce381d9a066 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.236920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 804/1605] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:02.237111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 802/1606] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:02 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:02.237229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bbf50-aea0-4779-a9db-1ce381d9a066 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.237323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680af31-80fa-4e15-aa8f-351844bbaf87 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.249850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bbf50-aea0-4779-a9db-1ce381d9a066 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.298241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680af31-80fa-4e15-aa8f-351844bbaf87 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.302912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbfdf08-0417-43c2-8313-e7d8cec61c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.303563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfdf08-0417-43c2-8313-e7d8cec61c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.303836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfdf08-0417-43c2-8313-e7d8cec61c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.304217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbfdf08-0417-43c2-8313-e7d8cec61c57 admin admin] PATH_INFO: `/v3/projects/b475d9ed19fb470aa81d133feadb658a/users/1b0bf21eb066497db842c72c63a95f4a/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.308122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa99f3-382c-4fd4-a598-608dddd86b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.308789 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa99f3-382c-4fd4-a598-608dddd86b30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.309050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa99f3-382c-4fd4-a598-608dddd86b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.309316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa99f3-382c-4fd4-a598-608dddd86b30 admin admin] PATH_INFO: `/v3/projects/fd5caa5782e74721a4c2991524182668/users/d23a498bde00483c8fdd68dcde975e5d/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.322571 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbfdf08-0417-43c2-8313-e7d8cec61c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b475d9ed19fb470aa81d133feadb658a, user_id=1b0bf21eb066497db842c72c63a95f4a, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.325227 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbfdf08-0417-43c2-8313-e7d8cec61c57 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.330118 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa99f3-382c-4fd4-a598-608dddd86b30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd5caa5782e74721a4c2991524182668, user_id=d23a498bde00483c8fdd68dcde975e5d, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.333331 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa99f3-382c-4fd4-a598-608dddd86b30 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.339706 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 803/1607] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:02 2022] PUT /identity/v3/projects/b475d9ed19fb470aa81d133feadb658a/users/1b0bf21eb066497db842c72c63a95f4a/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:02.340858 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbfdf08-0417-43c2-8313-e7d8cec61c57 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.357626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 805/1608] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:02 2022] PUT /identity/v3/projects/fd5caa5782e74721a4c2991524182668/users/d23a498bde00483c8fdd68dcde975e5d/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:02.361361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab780d-75f4-4c69-ab67-dc38e3541c86 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.361361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab780d-75f4-4c69-ab67-dc38e3541c86 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.361361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab780d-75f4-4c69-ab67-dc38e3541c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.388232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-15ab780d-75f4-4c69-ab67-dc38e3541c86 None None] MFA Rules not processed for user `f6525c9ecf4741f5a377126110bf6085`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:02.389446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-15ab780d-75f4-4c69-ab67-dc38e3541c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:02.405563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbfdf08-0417-43c2-8313-e7d8cec61c57 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.451947 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 806/1609] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:02.453796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e9b0f-6b87-49c7-9485-6a772d494241 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.454099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e9b0f-6b87-49c7-9485-6a772d494241 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.454381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e9b0f-6b87-49c7-9485-6a772d494241 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.455894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 807/1610] 10.222.0.55 () {54 vars in 912 bytes} [Tue Sep 27 17:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:24:02.456592 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e9b0f-6b87-49c7-9485-6a772d494241 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.465962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307d8a7-7cd6-4541-867d-708c56b85d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.467515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307d8a7-7cd6-4541-867d-708c56b85d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.468466 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307d8a7-7cd6-4541-867d-708c56b85d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.468896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307d8a7-7cd6-4541-867d-708c56b85d86 admin admin] PATH_INFO: `/v3/projects/e81bf8c903a44061be26a1059e9665cb/users/9300e254fc1549bc9569d4de78718afb/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.474054 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:02.490353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307d8a7-7cd6-4541-867d-708c56b85d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81bf8c903a44061be26a1059e9665cb, user_id=9300e254fc1549bc9569d4de78718afb, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.494315 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307d8a7-7cd6-4541-867d-708c56b85d86 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.508058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 804/1611] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:02 2022] PUT /identity/v3/projects/e81bf8c903a44061be26a1059e9665cb/users/9300e254fc1549bc9569d4de78718afb/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:02.508873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307d8a7-7cd6-4541-867d-708c56b85d86 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.520924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e9b0f-6b87-49c7-9485-6a772d494241 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.525563 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:02.572767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccc172-84c8-4834-8dc3-bc4e72384338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.573366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccc172-84c8-4834-8dc3-bc4e72384338 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.573642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccc172-84c8-4834-8dc3-bc4e72384338 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.573938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccc172-84c8-4834-8dc3-bc4e72384338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.574787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307d8a7-7cd6-4541-867d-708c56b85d86 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.580080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388524f-8b3f-4bec-8d0d-eb5e27666fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.580674 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388524f-8b3f-4bec-8d0d-eb5e27666fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.580942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388524f-8b3f-4bec-8d0d-eb5e27666fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.581204 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388524f-8b3f-4bec-8d0d-eb5e27666fa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.581823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388524f-8b3f-4bec-8d0d-eb5e27666fa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.583756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388524f-8b3f-4bec-8d0d-eb5e27666fa5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.583890 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccc172-84c8-4834-8dc3-bc4e72384338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.585371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccc172-84c8-4834-8dc3-bc4e72384338 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.595699 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 808/1612] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:02.596578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccc172-84c8-4834-8dc3-bc4e72384338 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.607759 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 805/1613] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:02 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:02.608093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccc172-84c8-4834-8dc3-bc4e72384338 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.609180 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77be018-1ed6-4c3e-82ff-d217f276370d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.609464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77be018-1ed6-4c3e-82ff-d217f276370d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.609748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77be018-1ed6-4c3e-82ff-d217f276370d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.613057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086ab30-cbbd-4da0-80ef-01a638d02a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.613767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086ab30-cbbd-4da0-80ef-01a638d02a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.614165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086ab30-cbbd-4da0-80ef-01a638d02a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.614421 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086ab30-cbbd-4da0-80ef-01a638d02a8a admin admin] PATH_INFO: `/v3/projects/37e99b06d94d4893963a88cd5333ec5b/users/1dde550901f04ea695b85db4e41d9628/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.630854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086ab30-cbbd-4da0-80ef-01a638d02a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=37e99b06d94d4893963a88cd5333ec5b, user_id=1dde550901f04ea695b85db4e41d9628, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.633536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086ab30-cbbd-4da0-80ef-01a638d02a8a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.644958 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b77be018-1ed6-4c3e-82ff-d217f276370d None None] MFA Rules not processed for user `1b0bf21eb066497db842c72c63a95f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:02.646298 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b77be018-1ed6-4c3e-82ff-d217f276370d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:02.648232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 809/1614] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:02 2022] PUT /identity/v3/projects/37e99b06d94d4893963a88cd5333ec5b/users/1dde550901f04ea695b85db4e41d9628/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:02.649356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9742b-59b2-4d98-8b27-2989c93b709c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.649776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9742b-59b2-4d98-8b27-2989c93b709c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.650053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9742b-59b2-4d98-8b27-2989c93b709c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.670325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-68b9742b-59b2-4d98-8b27-2989c93b709c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:02.671646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-68b9742b-59b2-4d98-8b27-2989c93b709c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:02.732755 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 806/1615] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:02.734112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b742d4-ce21-4119-8176-170956e8dd27 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.734394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b742d4-ce21-4119-8176-170956e8dd27 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.734668 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b742d4-ce21-4119-8176-170956e8dd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.745610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 810/1616] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:02.747318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e445cb-c7e9-4eb4-a170-721ec4ca2a0d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.747787 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e445cb-c7e9-4eb4-a170-721ec4ca2a0d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.748303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e445cb-c7e9-4eb4-a170-721ec4ca2a0d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.751093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 811/1617] 10.222.0.55 () {54 vars in 961 bytes} [Tue Sep 27 17:24:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:24:02.752684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9fc56-f9df-41a0-b5b1-22a2c2b77a5a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.753058 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9fc56-f9df-41a0-b5b1-22a2c2b77a5a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.753403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9fc56-f9df-41a0-b5b1-22a2c2b77a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.759977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-61b742d4-ce21-4119-8176-170956e8dd27 None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:02.761277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-61b742d4-ce21-4119-8176-170956e8dd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:02.791036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-edc9fc56-f9df-41a0-b5b1-22a2c2b77a5a None None] MFA Rules not processed for user `9300e254fc1549bc9569d4de78718afb`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:02.792248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-edc9fc56-f9df-41a0-b5b1-22a2c2b77a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:02.829556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:02.862886 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 807/1618] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:24:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:02.864291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b742d4-ce21-4119-8176-170956e8dd27 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.878118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b742d4-ce21-4119-8176-170956e8dd27 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.879850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 812/1619] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:02.881325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34752bf8-df9a-435d-a4ed-25d27f32d811 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.882119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34752bf8-df9a-435d-a4ed-25d27f32d811 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.882119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34752bf8-df9a-435d-a4ed-25d27f32d811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.888914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafdd73-27aa-4ee2-bd9f-b64073e994be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.889538 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafdd73-27aa-4ee2-bd9f-b64073e994be admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.889806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafdd73-27aa-4ee2-bd9f-b64073e994be admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.890088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acafdd73-27aa-4ee2-bd9f-b64073e994be admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.890743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafdd73-27aa-4ee2-bd9f-b64073e994be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:02.892343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acafdd73-27aa-4ee2-bd9f-b64073e994be admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:02.905056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 808/1620] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:02.905931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafdd73-27aa-4ee2-bd9f-b64073e994be admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.907811 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-acafdd73-27aa-4ee2-bd9f-b64073e994be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:02.910313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-34752bf8-df9a-435d-a4ed-25d27f32d811 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:02.911624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-34752bf8-df9a-435d-a4ed-25d27f32d811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:02.924293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 813/1621] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:02.924293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d887d-5065-4325-99f7-a02ffdf00c0a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.924293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d887d-5065-4325-99f7-a02ffdf00c0a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.924293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d887d-5065-4325-99f7-a02ffdf00c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:02.953612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-415d887d-5065-4325-99f7-a02ffdf00c0a None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:02.955753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-415d887d-5065-4325-99f7-a02ffdf00c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:02.975355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acafdd73-27aa-4ee2-bd9f-b64073e994be admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:02.977238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 814/1622] 10.222.0.55 () {58 vars in 1058 bytes} [Tue Sep 27 17:24:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:02.978375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d887d-5065-4325-99f7-a02ffdf00c0a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:02.984193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb335e-9593-4982-bc42-cf91d330e45a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:02.985792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:02.986242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb335e-9593-4982-bc42-cf91d330e45a service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:02.986513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb335e-9593-4982-bc42-cf91d330e45a service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:02.986821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb335e-9593-4982-bc42-cf91d330e45a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.003630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb335e-9593-4982-bc42-cf91d330e45a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:03.005201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb335e-9593-4982-bc42-cf91d330e45a service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:03.020321 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 809/1623] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:03.021756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb335e-9593-4982-bc42-cf91d330e45a service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:03.036267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb335e-9593-4982-bc42-cf91d330e45a service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:03.043226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf4da9-9750-4b2a-8565-3ef4454fe96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:03.044444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf4da9-9750-4b2a-8565-3ef4454fe96b admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.044765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf4da9-9750-4b2a-8565-3ef4454fe96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.045330 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bf4da9-9750-4b2a-8565-3ef4454fe96b admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.046014 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf4da9-9750-4b2a-8565-3ef4454fe96b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:03.048427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bf4da9-9750-4b2a-8565-3ef4454fe96b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:03.060991 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d887d-5065-4325-99f7-a02ffdf00c0a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:03.065266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12067ef-15d0-4873-b178-836a3bcab848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:03.065879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12067ef-15d0-4873-b178-836a3bcab848 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.066210 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12067ef-15d0-4873-b178-836a3bcab848 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.066643 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12067ef-15d0-4873-b178-836a3bcab848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.078471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12067ef-15d0-4873-b178-836a3bcab848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:03.079941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12067ef-15d0-4873-b178-836a3bcab848 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:03.084593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 810/1624] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:03 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:03.085412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf4da9-9750-4b2a-8565-3ef4454fe96b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:03.099816 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 815/1625] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:03.100847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12067ef-15d0-4873-b178-836a3bcab848 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:03.117200 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12067ef-15d0-4873-b178-836a3bcab848 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:03.163270 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bf4da9-9750-4b2a-8565-3ef4454fe96b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:03.170962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82119670-9657-418a-827e-e5bb06276936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:03.171714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82119670-9657-418a-827e-e5bb06276936 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.172062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82119670-9657-418a-827e-e5bb06276936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.172433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82119670-9657-418a-827e-e5bb06276936 admin admin] PATH_INFO: `/v3/projects/93b8558244294a49920234d44864e609/users/5758f956684e489d8990f65ab38b78ff/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.189640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82119670-9657-418a-827e-e5bb06276936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93b8558244294a49920234d44864e609, user_id=5758f956684e489d8990f65ab38b78ff, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:03.193212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442202ad-fb91-4663-924e-d17dc0a80514 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:03.193374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82119670-9657-418a-827e-e5bb06276936 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:03.193873 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442202ad-fb91-4663-924e-d17dc0a80514 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.194329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442202ad-fb91-4663-924e-d17dc0a80514 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.194848 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442202ad-fb91-4663-924e-d17dc0a80514 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.209464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442202ad-fb91-4663-924e-d17dc0a80514 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:03.211223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442202ad-fb91-4663-924e-d17dc0a80514 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:03.212656 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 811/1626] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:03 2022] PUT /identity/v3/projects/93b8558244294a49920234d44864e609/users/5758f956684e489d8990f65ab38b78ff/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:03.213969 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82119670-9657-418a-827e-e5bb06276936 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:03.228943 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 816/1627] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:03.229941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442202ad-fb91-4663-924e-d17dc0a80514 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:03.242716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442202ad-fb91-4663-924e-d17dc0a80514 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:03.297922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:03.342191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707bf08-4063-4a6e-a851-29937d750f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:03.343080 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707bf08-4063-4a6e-a851-29937d750f25 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.343408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707bf08-4063-4a6e-a851-29937d750f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.343706 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82119670-9657-418a-827e-e5bb06276936 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:03.343910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707bf08-4063-4a6e-a851-29937d750f25 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.344691 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707bf08-4063-4a6e-a851-29937d750f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:03.347595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707bf08-4063-4a6e-a851-29937d750f25 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:03.351793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451207d5-534d-4ead-bc2e-7d452308a64f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:03.352759 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451207d5-534d-4ead-bc2e-7d452308a64f service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.353248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451207d5-534d-4ead-bc2e-7d452308a64f service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.353522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451207d5-534d-4ead-bc2e-7d452308a64f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.376432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 817/1628] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:03 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:03.377935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707bf08-4063-4a6e-a851-29937d750f25 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:03.391582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707bf08-4063-4a6e-a851-29937d750f25 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:03.398545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd2cc8-073d-4de9-83a1-f4e9d782a47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:03.399697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd2cc8-073d-4de9-83a1-f4e9d782a47f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.400270 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd2cc8-073d-4de9-83a1-f4e9d782a47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.400849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd2cc8-073d-4de9-83a1-f4e9d782a47f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.401908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd2cc8-073d-4de9-83a1-f4e9d782a47f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:03.404218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd2cc8-073d-4de9-83a1-f4e9d782a47f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:03.418374 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 818/1629] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:03.420462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd2cc8-073d-4de9-83a1-f4e9d782a47f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:03.435172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451207d5-534d-4ead-bc2e-7d452308a64f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:03.437278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451207d5-534d-4ead-bc2e-7d452308a64f service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:03.439752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd2cc8-073d-4de9-83a1-f4e9d782a47f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:03.447892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815bc4f-1f74-47d4-97d2-f2b1614be01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:03.449105 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815bc4f-1f74-47d4-97d2-f2b1614be01e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.449381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 812/1630] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:03.450260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451207d5-534d-4ead-bc2e-7d452308a64f service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:03.462179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451207d5-534d-4ead-bc2e-7d452308a64f service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:03.462739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815bc4f-1f74-47d4-97d2-f2b1614be01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.463171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815bc4f-1f74-47d4-97d2-f2b1614be01e admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.463903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815bc4f-1f74-47d4-97d2-f2b1614be01e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:03.467718 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329292b-3006-4e06-8e2f-dc4ea91de2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:03.467943 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815bc4f-1f74-47d4-97d2-f2b1614be01e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:03.468422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329292b-3006-4e06-8e2f-dc4ea91de2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.468698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329292b-3006-4e06-8e2f-dc4ea91de2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.468966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329292b-3006-4e06-8e2f-dc4ea91de2dc admin admin] PATH_INFO: `/v3/projects/93b8558244294a49920234d44864e609/users/5758f956684e489d8990f65ab38b78ff/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.471362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329292b-3006-4e06-8e2f-dc4ea91de2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=93b8558244294a49920234d44864e609, user_id=5758f956684e489d8990f65ab38b78ff, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:03.476508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329292b-3006-4e06-8e2f-dc4ea91de2dc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:03.494097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 819/1632] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:03.495121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815bc4f-1f74-47d4-97d2-f2b1614be01e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:03.499364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 813/1632] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:03 2022] PUT /identity/v3/projects/93b8558244294a49920234d44864e609/users/5758f956684e489d8990f65ab38b78ff/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:03.500856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f773d24-26f7-4ec3-8203-755c87f16462 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.501168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f773d24-26f7-4ec3-8203-755c87f16462 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.501440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f773d24-26f7-4ec3-8203-755c87f16462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.514909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815bc4f-1f74-47d4-97d2-f2b1614be01e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:03.537587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0f773d24-26f7-4ec3-8203-755c87f16462 None None] MFA Rules not processed for user `5758f956684e489d8990f65ab38b78ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:03.539053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0f773d24-26f7-4ec3-8203-755c87f16462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:03.547992 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:03.629741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f42c2a-607c-4626-85c9-d2bc8cb116a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:03.630620 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f42c2a-607c-4626-85c9-d2bc8cb116a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.631152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f42c2a-607c-4626-85c9-d2bc8cb116a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.631895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f42c2a-607c-4626-85c9-d2bc8cb116a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.633022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f42c2a-607c-4626-85c9-d2bc8cb116a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:03.635189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f42c2a-607c-4626-85c9-d2bc8cb116a2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:03.649770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 820/1633] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:03.654731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f42c2a-607c-4626-85c9-d2bc8cb116a2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:03.664090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 814/1634] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:03.678136 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f773d24-26f7-4ec3-8203-755c87f16462 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:03.680436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f42c2a-607c-4626-85c9-d2bc8cb116a2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:03.689994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30b841-3890-48fe-9460-f11f8b7b0863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:03.691232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30b841-3890-48fe-9460-f11f8b7b0863 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.691566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30b841-3890-48fe-9460-f11f8b7b0863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.692002 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30b841-3890-48fe-9460-f11f8b7b0863 admin admin] PATH_INFO: `/v3/projects/3a7cbd60b036481ea69455bd5bea1a27/users/70bbfc114c314b8d837c16a8cb9b9712/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.719478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30b841-3890-48fe-9460-f11f8b7b0863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a7cbd60b036481ea69455bd5bea1a27, user_id=70bbfc114c314b8d837c16a8cb9b9712, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:03.724913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30b841-3890-48fe-9460-f11f8b7b0863 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:03.762609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 821/1635] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:03 2022] PUT /identity/v3/projects/3a7cbd60b036481ea69455bd5bea1a27/users/70bbfc114c314b8d837c16a8cb9b9712/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:03.771001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca077dd7-3392-4398-ad58-5e7a845d1d0f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.771466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca077dd7-3392-4398-ad58-5e7a845d1d0f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.771982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca077dd7-3392-4398-ad58-5e7a845d1d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.772853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f773d24-26f7-4ec3-8203-755c87f16462 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:03.795639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:03.824217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ca077dd7-3392-4398-ad58-5e7a845d1d0f None None] MFA Rules not processed for user `70bbfc114c314b8d837c16a8cb9b9712`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:03.826530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ca077dd7-3392-4398-ad58-5e7a845d1d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:03.879850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e252c-4469-4539-8de1-bb2fa1f58841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:03.880580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e252c-4469-4539-8de1-bb2fa1f58841 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:03.880862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e252c-4469-4539-8de1-bb2fa1f58841 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:03.881134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e252c-4469-4539-8de1-bb2fa1f58841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:03.934014 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 822/1636] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:24:03.935774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca077dd7-3392-4398-ad58-5e7a845d1d0f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:04.028181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e252c-4469-4539-8de1-bb2fa1f58841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:04.032405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e252c-4469-4539-8de1-bb2fa1f58841 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:04.084281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 815/1637] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:24:04.087918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e252c-4469-4539-8de1-bb2fa1f58841 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:04.132520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca077dd7-3392-4398-ad58-5e7a845d1d0f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:04.161120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcbf78-d271-40c3-88c8-b4d0531fb5de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:04.161778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcbf78-d271-40c3-88c8-b4d0531fb5de service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:04.167165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcbf78-d271-40c3-88c8-b4d0531fb5de service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:04.167165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcbf78-d271-40c3-88c8-b4d0531fb5de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:04.219548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:04.316380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e252c-4469-4539-8de1-bb2fa1f58841 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:04.337620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77aa022-805a-4fbb-a408-7da175b24850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:04.337620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77aa022-805a-4fbb-a408-7da175b24850 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:04.337620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77aa022-805a-4fbb-a408-7da175b24850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:04.337620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77aa022-805a-4fbb-a408-7da175b24850 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:04.338855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77aa022-805a-4fbb-a408-7da175b24850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:04.345875 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77aa022-805a-4fbb-a408-7da175b24850 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:04.367236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcbf78-d271-40c3-88c8-b4d0531fb5de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:04.368868 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcbf78-d271-40c3-88c8-b4d0531fb5de service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:04.389329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 823/1638] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:04.426925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 816/1639] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:04 2022] POST /identity/v3/projects => generated 377 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:04.431292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcbf78-d271-40c3-88c8-b4d0531fb5de service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:04.452023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcbf78-d271-40c3-88c8-b4d0531fb5de service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:04.460102 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a39a9-d073-4928-9d17-0a0fcc76aa1e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:04.460374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a39a9-d073-4928-9d17-0a0fcc76aa1e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:04.460700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a39a9-d073-4928-9d17-0a0fcc76aa1e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:04.462600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 817/1640] 10.222.0.55 () {54 vars in 912 bytes} [Tue Sep 27 17:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:24:04.468717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ab8ea-fac0-4aa3-8e52-309095519019 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:04.469042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ab8ea-fac0-4aa3-8e52-309095519019 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:04.469339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ab8ea-fac0-4aa3-8e52-309095519019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:04.525072 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f81ab8ea-fac0-4aa3-8e52-309095519019 None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:04.526844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f81ab8ea-fac0-4aa3-8e52-309095519019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:04.656337 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:04.681088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe72b05-771a-4b68-ad7b-b4ebc7efaa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:04.682902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe72b05-771a-4b68-ad7b-b4ebc7efaa45 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:04.683316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe72b05-771a-4b68-ad7b-b4ebc7efaa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:04.683721 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe72b05-771a-4b68-ad7b-b4ebc7efaa45 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:04.684680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe72b05-771a-4b68-ad7b-b4ebc7efaa45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:04.688282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe72b05-771a-4b68-ad7b-b4ebc7efaa45 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:04.728930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 818/1641] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:24:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:24:04.738668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 824/1642] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:04 2022] POST /identity/v3/users => generated 295 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:04.740990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe72b05-771a-4b68-ad7b-b4ebc7efaa45 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:04.744111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ab8ea-fac0-4aa3-8e52-309095519019 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:04.760572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe72b05-771a-4b68-ad7b-b4ebc7efaa45 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:04.769421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ab8ea-fac0-4aa3-8e52-309095519019 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:04.777029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:04.777694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:04.777998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:04.778339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] PATH_INFO: `/v3/users/f6525c9ecf4741f5a377126110bf6085` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:04.779717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68f3fb-7c89-42b8-8c11-5915f29eb39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:04.780339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68f3fb-7c89-42b8-8c11-5915f29eb39f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:04.780725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68f3fb-7c89-42b8-8c11-5915f29eb39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:04.780896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68f3fb-7c89-42b8-8c11-5915f29eb39f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:04.782166 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68f3fb-7c89-42b8-8c11-5915f29eb39f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:04.783904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68f3fb-7c89-42b8-8c11-5915f29eb39f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:04.792116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6525c9ecf4741f5a377126110bf6085)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:04.797562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:04.814303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 825/1643] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:04.815711 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68f3fb-7c89-42b8-8c11-5915f29eb39f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:04.949435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6525c9ecf4741f5a377126110bf6085'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:04.970465 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6525c9ecf4741f5a377126110bf6085'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:04.971602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6525c9ecf4741f5a377126110bf6085'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:04.987867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6525c9ecf4741f5a377126110bf6085'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:05.009781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6525c9ecf4741f5a377126110bf6085'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:05.020401 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68f3fb-7c89-42b8-8c11-5915f29eb39f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:05.025649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 819/1644] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:04 2022] DELETE /identity/v3/users/f6525c9ecf4741f5a377126110bf6085 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:05.026605 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:05.031771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:05.179729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07eb10d-c518-4ace-ad01-2a782d001687 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:05.195178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5f815-212c-44b8-886e-8eff03e1bd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:05.197179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5f815-212c-44b8-886e-8eff03e1bd22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:05.197615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5f815-212c-44b8-886e-8eff03e1bd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:05.197791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9cb6e-c9e6-4a64-b356-f1b178ae6a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:05.199854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5f815-212c-44b8-886e-8eff03e1bd22 admin admin] PATH_INFO: `/v3/projects/69840972df1f48e58f12c14c021d5009/users/2ee54bde58874323accf29efa724e0d1/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:05.200099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9cb6e-c9e6-4a64-b356-f1b178ae6a00 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:05.200495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9cb6e-c9e6-4a64-b356-f1b178ae6a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:05.201766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed9cb6e-c9e6-4a64-b356-f1b178ae6a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:05.206158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5ed9cb6e-c9e6-4a64-b356-f1b178ae6a00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:05.218872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:05.249830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5f815-212c-44b8-886e-8eff03e1bd22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69840972df1f48e58f12c14c021d5009, user_id=2ee54bde58874323accf29efa724e0d1, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:05.256545 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5f815-212c-44b8-886e-8eff03e1bd22 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:05.287209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 820/1645] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:05 2022] PUT /identity/v3/projects/69840972df1f48e58f12c14c021d5009/users/2ee54bde58874323accf29efa724e0d1/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:05.288798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e60bf-ee8e-4722-81bc-7a6414eff4cb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:05.288981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e60bf-ee8e-4722-81bc-7a6414eff4cb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:05.289393 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e60bf-ee8e-4722-81bc-7a6414eff4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:05.343400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-972e60bf-ee8e-4722-81bc-7a6414eff4cb None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:05.345505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-972e60bf-ee8e-4722-81bc-7a6414eff4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:05.428149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed9cb6e-c9e6-4a64-b356-f1b178ae6a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:05.428606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:05.433498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed9cb6e-c9e6-4a64-b356-f1b178ae6a00 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:05.472664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 826/1646] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:05.476822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9cb6e-c9e6-4a64-b356-f1b178ae6a00 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:05.482309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5ed9cb6e-c9e6-4a64-b356-f1b178ae6a00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:05.491516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 821/1647] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:05.493130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e60bf-ee8e-4722-81bc-7a6414eff4cb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:05.520319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e60bf-ee8e-4722-81bc-7a6414eff4cb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:05.532189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28359fd-b793-4445-907e-b39deed64930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:05.532441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed9cb6e-c9e6-4a64-b356-f1b178ae6a00 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:05.533046 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28359fd-b793-4445-907e-b39deed64930 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:05.533443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28359fd-b793-4445-907e-b39deed64930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:05.533813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28359fd-b793-4445-907e-b39deed64930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:05.534650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28359fd-b793-4445-907e-b39deed64930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:05.537904 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28359fd-b793-4445-907e-b39deed64930 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:05.549269 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448eb72-62d8-49e8-81bc-61be2d57872d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:05.551490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448eb72-62d8-49e8-81bc-61be2d57872d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:05.553447 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448eb72-62d8-49e8-81bc-61be2d57872d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:05.554582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448eb72-62d8-49e8-81bc-61be2d57872d admin admin] PATH_INFO: `/v3/users/c3bc72fa2ee34c268308bce70122babc` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:05.560919 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448eb72-62d8-49e8-81bc-61be2d57872d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3bc72fa2ee34c268308bce70122babc)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:05.566660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 822/1648] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:05.568510 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28359fd-b793-4445-907e-b39deed64930 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:05.570788 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-e448eb72-62d8-49e8-81bc-61be2d57872d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:24:05.570788 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 27 17:24:05.570788 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: Sep 27 17:24:05.572001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448eb72-62d8-49e8-81bc-61be2d57872d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:05.576859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c28359fd-b793-4445-907e-b39deed64930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:05.674533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28359fd-b793-4445-907e-b39deed64930 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:05.680874 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c44b1-67d3-47f3-8e29-9320ecd47cf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:05.682328 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c44b1-67d3-47f3-8e29-9320ecd47cf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:05.682947 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c44b1-67d3-47f3-8e29-9320ecd47cf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:05.684066 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c44b1-67d3-47f3-8e29-9320ecd47cf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:05.707953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c44b1-67d3-47f3-8e29-9320ecd47cf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:05.710888 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c44b1-67d3-47f3-8e29-9320ecd47cf4 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:05.730723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 823/1649] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:05.732856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c44b1-67d3-47f3-8e29-9320ecd47cf4 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:05.737844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e448eb72-62d8-49e8-81bc-61be2d57872d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3bc72fa2ee34c268308bce70122babc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:05.738738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e448eb72-62d8-49e8-81bc-61be2d57872d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3bc72fa2ee34c268308bce70122babc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:05.745970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c44b1-67d3-47f3-8e29-9320ecd47cf4 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:05.760681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933273e7-0e6b-4291-96f2-5d4966722694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:05.762684 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933273e7-0e6b-4291-96f2-5d4966722694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:05.763774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933273e7-0e6b-4291-96f2-5d4966722694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:05.764267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933273e7-0e6b-4291-96f2-5d4966722694 admin admin] PATH_INFO: `/v3/projects/69840972df1f48e58f12c14c021d5009/users/2ee54bde58874323accf29efa724e0d1/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:05.768235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933273e7-0e6b-4291-96f2-5d4966722694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69840972df1f48e58f12c14c021d5009, user_id=2ee54bde58874323accf29efa724e0d1, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:05.776678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933273e7-0e6b-4291-96f2-5d4966722694 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:05.780173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e448eb72-62d8-49e8-81bc-61be2d57872d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3bc72fa2ee34c268308bce70122babc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:05.796679 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e448eb72-62d8-49e8-81bc-61be2d57872d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3bc72fa2ee34c268308bce70122babc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:05.800681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 824/1650] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:05 2022] PUT /identity/v3/projects/69840972df1f48e58f12c14c021d5009/users/2ee54bde58874323accf29efa724e0d1/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:05.815765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5a8b2-de93-4331-957a-d64f5efccffc None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:05.816319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5a8b2-de93-4331-957a-d64f5efccffc None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:05.816731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5a8b2-de93-4331-957a-d64f5efccffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:05.861964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e448eb72-62d8-49e8-81bc-61be2d57872d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3bc72fa2ee34c268308bce70122babc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:05.879288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 827/1651] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:05 2022] DELETE /identity/v3/users/c3bc72fa2ee34c268308bce70122babc => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:05.887712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac6041-041d-4cde-aa57-b5dc4b6a02e7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:05.888550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac6041-041d-4cde-aa57-b5dc4b6a02e7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:05.889108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac6041-041d-4cde-aa57-b5dc4b6a02e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:05.940894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c4e5a8b2-de93-4331-957a-d64f5efccffc None None] MFA Rules not processed for user `2ee54bde58874323accf29efa724e0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:05.945702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c4e5a8b2-de93-4331-957a-d64f5efccffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:05.956754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9bac6041-041d-4cde-aa57-b5dc4b6a02e7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:05.962387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9bac6041-041d-4cde-aa57-b5dc4b6a02e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:06.155047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 828/1652] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:06.160406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 825/1653] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:06.173849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac6041-041d-4cde-aa57-b5dc4b6a02e7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:06.175776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9bac6041-041d-4cde-aa57-b5dc4b6a02e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:06.186906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:06.273550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5a8b2-de93-4331-957a-d64f5efccffc None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:06.273550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c4e5a8b2-de93-4331-957a-d64f5efccffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:06.349820 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac6041-041d-4cde-aa57-b5dc4b6a02e7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:06.362115 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fc6f5-c020-414f-97b1-e74e6b5e1411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:06.362968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fc6f5-c020-414f-97b1-e74e6b5e1411 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:06.363404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fc6f5-c020-414f-97b1-e74e6b5e1411 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:06.363777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fc6f5-c020-414f-97b1-e74e6b5e1411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:06.385687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fc6f5-c020-414f-97b1-e74e6b5e1411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:06.387555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fc6f5-c020-414f-97b1-e74e6b5e1411 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:06.408594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 829/1654] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:06.423275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:06.457733 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5a8b2-de93-4331-957a-d64f5efccffc None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:06.467920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a5899-2ce0-4566-a55e-16fb793e9f79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:06.470300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a5899-2ce0-4566-a55e-16fb793e9f79 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:06.471705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a5899-2ce0-4566-a55e-16fb793e9f79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:06.472536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a5899-2ce0-4566-a55e-16fb793e9f79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:06.498167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a5899-2ce0-4566-a55e-16fb793e9f79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:06.499915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a5899-2ce0-4566-a55e-16fb793e9f79 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:06.524889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 826/1655] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:06.546166 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fc6f5-c020-414f-97b1-e74e6b5e1411 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:06.555893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fc6f5-c020-414f-97b1-e74e6b5e1411 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:06.561372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93750889-1d8f-4223-bd5a-646f21a9ef12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:06.562795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93750889-1d8f-4223-bd5a-646f21a9ef12 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:06.563357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93750889-1d8f-4223-bd5a-646f21a9ef12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:06.564195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93750889-1d8f-4223-bd5a-646f21a9ef12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:06.566494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-93750889-1d8f-4223-bd5a-646f21a9ef12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:06.738108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93750889-1d8f-4223-bd5a-646f21a9ef12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:06.742428 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93750889-1d8f-4223-bd5a-646f21a9ef12 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:06.781445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 830/1656] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:07.619179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a5899-2ce0-4566-a55e-16fb793e9f79 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:07.622660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c38a5899-2ce0-4566-a55e-16fb793e9f79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:07.654262 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a5899-2ce0-4566-a55e-16fb793e9f79 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:07.676876 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c902a-ceda-4bd4-ada1-d5cad43138c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:07.677840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c902a-ceda-4bd4-ada1-d5cad43138c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:07.678419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c902a-ceda-4bd4-ada1-d5cad43138c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:07.678882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c902a-ceda-4bd4-ada1-d5cad43138c3 admin admin] PATH_INFO: `/v3/projects/94de95e80dbc4006b100d487e0f0921c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:07.681132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c902a-ceda-4bd4-ada1-d5cad43138c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94de95e80dbc4006b100d487e0f0921c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:07.684170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-360c902a-ceda-4bd4-ada1-d5cad43138c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 27 17:24:07.684170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 27 17:24:07.684170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: Sep 27 17:24:07.689274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c902a-ceda-4bd4-ada1-d5cad43138c3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:07.853505 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93750889-1d8f-4223-bd5a-646f21a9ef12 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:07.860704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-93750889-1d8f-4223-bd5a-646f21a9ef12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:07.932285 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93750889-1d8f-4223-bd5a-646f21a9ef12 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:07.938909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-360c902a-ceda-4bd4-ada1-d5cad43138c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94de95e80dbc4006b100d487e0f0921c', 'request_id': 'req-360c902a-ceda-4bd4-ada1-d5cad43138c3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:07.954046 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-360c902a-ceda-4bd4-ada1-d5cad43138c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94de95e80dbc4006b100d487e0f0921c', 'request_id': 'req-360c902a-ceda-4bd4-ada1-d5cad43138c3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:07.989323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-360c902a-ceda-4bd4-ada1-d5cad43138c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94de95e80dbc4006b100d487e0f0921c', 'request_id': 'req-360c902a-ceda-4bd4-ada1-d5cad43138c3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:08.015560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-360c902a-ceda-4bd4-ada1-d5cad43138c3 admin admin] The token cache is being invalidate because project 94de95e80dbc4006b100d487e0f0921c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:08.015960 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-360c902a-ceda-4bd4-ada1-d5cad43138c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:08.016859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-360c902a-ceda-4bd4-ada1-d5cad43138c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:08.032123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 827/1657] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:07 2022] DELETE /identity/v3/projects/94de95e80dbc4006b100d487e0f0921c => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:08.033992 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc5620-f097-4c24-bc22-aa0aeb63449a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:08.034501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc5620-f097-4c24-bc22-aa0aeb63449a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:08.035215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc5620-f097-4c24-bc22-aa0aeb63449a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:08.080694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:08.081257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:08.081558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:08.081826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] PATH_INFO: `/v3/projects/63e37b343da749f7a43804b2d1224288` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:08.083325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e37b343da749f7a43804b2d1224288)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:08.087979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6ccc5620-f097-4c24-bc22-aa0aeb63449a None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:08.089386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6ccc5620-f097-4c24-bc22-aa0aeb63449a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:08.090203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:08.118112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 828/1658] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:08.127433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f07a2b-3989-4259-b2e7-d89baa0f6295 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:08.127715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f07a2b-3989-4259-b2e7-d89baa0f6295 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:08.127993 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f07a2b-3989-4259-b2e7-d89baa0f6295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:08.152142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-79f07a2b-3989-4259-b2e7-d89baa0f6295 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:08.153398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-79f07a2b-3989-4259-b2e7-d89baa0f6295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:08.177355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 829/1659] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:08.179093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f07a2b-3989-4259-b2e7-d89baa0f6295 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:08.198556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f07a2b-3989-4259-b2e7-d89baa0f6295 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:08.206743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:08.210657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e37b343da749f7a43804b2d1224288', 'request_id': 'req-c639fb52-0f1e-4bc5-8b5a-926a8301f763'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:08.225240 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e37b343da749f7a43804b2d1224288', 'request_id': 'req-c639fb52-0f1e-4bc5-8b5a-926a8301f763'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:08.238000 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e37b343da749f7a43804b2d1224288', 'request_id': 'req-c639fb52-0f1e-4bc5-8b5a-926a8301f763'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:08.268457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] The token cache is being invalidate because project 63e37b343da749f7a43804b2d1224288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:08.269178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:08.270095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:08.273017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 831/1660] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:07 2022] DELETE /identity/v3/projects/63e37b343da749f7a43804b2d1224288 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:24:08.277065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:08.279472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:08.309228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460a1d0-0cf2-4058-95a1-5fafa9dafaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:08.311707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460a1d0-0cf2-4058-95a1-5fafa9dafaf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:08.311996 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460a1d0-0cf2-4058-95a1-5fafa9dafaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:08.312273 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0460a1d0-0cf2-4058-95a1-5fafa9dafaf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:08.315947 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460a1d0-0cf2-4058-95a1-5fafa9dafaf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:08.316255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460a1d0-0cf2-4058-95a1-5fafa9dafaf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:08.318287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0460a1d0-0cf2-4058-95a1-5fafa9dafaf0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:08.347550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 830/1661] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:24:08.348514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460a1d0-0cf2-4058-95a1-5fafa9dafaf0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:08.353367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0460a1d0-0cf2-4058-95a1-5fafa9dafaf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:08.387026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0460a1d0-0cf2-4058-95a1-5fafa9dafaf0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:08.400912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48367e-9075-4bd0-a122-5b2db52d9ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:08.401556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48367e-9075-4bd0-a122-5b2db52d9ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:08.401858 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48367e-9075-4bd0-a122-5b2db52d9ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:08.407116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48367e-9075-4bd0-a122-5b2db52d9ad2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:08.408971 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48367e-9075-4bd0-a122-5b2db52d9ad2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:08.414206 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48367e-9075-4bd0-a122-5b2db52d9ad2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:08.441539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639fb52-0f1e-4bc5-8b5a-926a8301f763 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:08.455745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:08.456399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:08.456675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:08.457037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] PATH_INFO: `/v3/users/5758f956684e489d8990f65ab38b78ff` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:08.458607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5758f956684e489d8990f65ab38b78ff)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:08.462881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:08.497647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 831/1662] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:08 2022] POST /identity/v3/projects => generated 363 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:08.500837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48367e-9075-4bd0-a122-5b2db52d9ad2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:08.611193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:08.621558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5758f956684e489d8990f65ab38b78ff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:08.628634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5758f956684e489d8990f65ab38b78ff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:08.640935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5758f956684e489d8990f65ab38b78ff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:08.673108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5758f956684e489d8990f65ab38b78ff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:08.695640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5758f956684e489d8990f65ab38b78ff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:08.703535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 832/1663] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:08 2022] DELETE /identity/v3/users/5758f956684e489d8990f65ab38b78ff => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:24:08.724593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48367e-9075-4bd0-a122-5b2db52d9ad2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:08.873499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e48458-9015-425b-a94c-e1652ae716d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:08.875085 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e48458-9015-425b-a94c-e1652ae716d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:08.875657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e48458-9015-425b-a94c-e1652ae716d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:08.876399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e48458-9015-425b-a94c-e1652ae716d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:08.877146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e48458-9015-425b-a94c-e1652ae716d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:08.881257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e48458-9015-425b-a94c-e1652ae716d6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:08.957602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 832/1664] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:08 2022] POST /identity/v3/users => generated 282 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:08.959098 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:08.960658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:08.991997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115df001-4f6e-43f1-85d5-18188199b85a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:09.009607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a52b6-2b64-41e6-9636-2f37f2797c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:09.010455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a52b6-2b64-41e6-9636-2f37f2797c6e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:09.010817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a52b6-2b64-41e6-9636-2f37f2797c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:09.011198 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a52b6-2b64-41e6-9636-2f37f2797c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:09.012099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a52b6-2b64-41e6-9636-2f37f2797c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:09.018529 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a52b6-2b64-41e6-9636-2f37f2797c6e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:09.058625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 833/1665] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:09.060236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e48458-9015-425b-a94c-e1652ae716d6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:09.089228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e48458-9015-425b-a94c-e1652ae716d6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:09.109645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54454a11-ec16-458b-b235-3ea236f86be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:09.110361 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54454a11-ec16-458b-b235-3ea236f86be2 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:09.110618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54454a11-ec16-458b-b235-3ea236f86be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:09.110885 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54454a11-ec16-458b-b235-3ea236f86be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:09.111588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54454a11-ec16-458b-b235-3ea236f86be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:09.113246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54454a11-ec16-458b-b235-3ea236f86be2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:09.150778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a52b6-2b64-41e6-9636-2f37f2797c6e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:09.159848 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 833/1666] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:09.171328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a52b6-2b64-41e6-9636-2f37f2797c6e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:09.184180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e453b8-b5d2-49b4-8884-3f2ae30eb9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:09.185304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e453b8-b5d2-49b4-8884-3f2ae30eb9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:09.185595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e453b8-b5d2-49b4-8884-3f2ae30eb9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:09.185861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e453b8-b5d2-49b4-8884-3f2ae30eb9bc admin admin] PATH_INFO: `/v3/projects/31f037b8adec484fa2e6756ec0ced4d2/users/92e7560512b342d8a8e5330bc27d463f/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:09.229172 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e453b8-b5d2-49b4-8884-3f2ae30eb9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=31f037b8adec484fa2e6756ec0ced4d2, user_id=92e7560512b342d8a8e5330bc27d463f, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:09.232657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e453b8-b5d2-49b4-8884-3f2ae30eb9bc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:09.258780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 834/1667] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:09 2022] PUT /identity/v3/projects/31f037b8adec484fa2e6756ec0ced4d2/users/92e7560512b342d8a8e5330bc27d463f/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:09.270420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9c08c-4404-422b-aeb5-26e2402b7381 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:09.270711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9c08c-4404-422b-aeb5-26e2402b7381 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:09.271097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9c08c-4404-422b-aeb5-26e2402b7381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:09.359975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-29a9c08c-4404-422b-aeb5-26e2402b7381 None None] MFA Rules not processed for user `92e7560512b342d8a8e5330bc27d463f`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:09.361248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-29a9c08c-4404-422b-aeb5-26e2402b7381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:09.547970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 834/1668] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:09.552359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e453b8-b5d2-49b4-8884-3f2ae30eb9bc admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:09.608944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:09.702524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e453b8-b5d2-49b4-8884-3f2ae30eb9bc admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:09.711986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a623-6451-4283-8614-78e82c7bf00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:09.713198 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a623-6451-4283-8614-78e82c7bf00a admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:09.713552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a623-6451-4283-8614-78e82c7bf00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:09.713899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842a623-6451-4283-8614-78e82c7bf00a admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:09.714655 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842a623-6451-4283-8614-78e82c7bf00a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:09.718156 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842a623-6451-4283-8614-78e82c7bf00a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:09.761405 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 835/1669] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:09 2022] POST /identity/v3/projects => generated 365 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:09.765861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9c08c-4404-422b-aeb5-26e2402b7381 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:09.791717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9c08c-4404-422b-aeb5-26e2402b7381 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:09.881317 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:09.926834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ab176-e378-43a2-89be-2aeb3f8d7993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:09.927503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ab176-e378-43a2-89be-2aeb3f8d7993 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:09.927777 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ab176-e378-43a2-89be-2aeb3f8d7993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:09.928058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ab176-e378-43a2-89be-2aeb3f8d7993 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:09.928748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ab176-e378-43a2-89be-2aeb3f8d7993 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:09.931654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ab176-e378-43a2-89be-2aeb3f8d7993 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:10.002658 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 835/1670] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:09 2022] POST /identity/v3/users => generated 289 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:10.006867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a623-6451-4283-8614-78e82c7bf00a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:10.033926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842a623-6451-4283-8614-78e82c7bf00a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:10.060322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677de92-530e-4795-a71d-0743cdf7ebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:10.062046 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677de92-530e-4795-a71d-0743cdf7ebe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:10.062458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677de92-530e-4795-a71d-0743cdf7ebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:10.062895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677de92-530e-4795-a71d-0743cdf7ebe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:10.063974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677de92-530e-4795-a71d-0743cdf7ebe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:10.070288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677de92-530e-4795-a71d-0743cdf7ebe5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:10.121594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 836/1671] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:10.122808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ab176-e378-43a2-89be-2aeb3f8d7993 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:10.149040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ab176-e378-43a2-89be-2aeb3f8d7993 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:10.160023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9682d2-b29c-40ed-b91e-21390a1e20cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:10.160912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9682d2-b29c-40ed-b91e-21390a1e20cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:10.161194 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9682d2-b29c-40ed-b91e-21390a1e20cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:10.161470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9682d2-b29c-40ed-b91e-21390a1e20cc admin admin] PATH_INFO: `/v3/projects/1dfdf2a73d98489bbf15b87ff309b922/users/b7de04d272d5496d88c115a4f2304ed4/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:10.203350 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9682d2-b29c-40ed-b91e-21390a1e20cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dfdf2a73d98489bbf15b87ff309b922, user_id=b7de04d272d5496d88c115a4f2304ed4, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:10.206035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9682d2-b29c-40ed-b91e-21390a1e20cc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:10.260048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 836/1672] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:10 2022] PUT /identity/v3/projects/1dfdf2a73d98489bbf15b87ff309b922/users/b7de04d272d5496d88c115a4f2304ed4/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:10.261284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677de92-530e-4795-a71d-0743cdf7ebe5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:10.281803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677de92-530e-4795-a71d-0743cdf7ebe5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:10.400076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9682d2-b29c-40ed-b91e-21390a1e20cc admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:10.401992 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1d9682d2-b29c-40ed-b91e-21390a1e20cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:10.465651 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec47ad-9454-444d-a574-4812e3df42c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:10.467009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec47ad-9454-444d-a574-4812e3df42c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:10.467289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec47ad-9454-444d-a574-4812e3df42c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:10.467561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec47ad-9454-444d-a574-4812e3df42c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:10.468213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec47ad-9454-444d-a574-4812e3df42c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:10.469829 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec47ad-9454-444d-a574-4812e3df42c5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:10.527177 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 837/1673] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:10.527932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec47ad-9454-444d-a574-4812e3df42c5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:10.558802 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec47ad-9454-444d-a574-4812e3df42c5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:10.573795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a8edac-a861-4592-a5e8-a2eb33a2a693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:10.576150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8edac-a861-4592-a5e8-a2eb33a2a693 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:10.577171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8edac-a861-4592-a5e8-a2eb33a2a693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:10.579033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8edac-a861-4592-a5e8-a2eb33a2a693 admin admin] PATH_INFO: `/v3/projects/1dfdf2a73d98489bbf15b87ff309b922/users/b7de04d272d5496d88c115a4f2304ed4/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:10.582979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8edac-a861-4592-a5e8-a2eb33a2a693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dfdf2a73d98489bbf15b87ff309b922, user_id=b7de04d272d5496d88c115a4f2304ed4, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:10.590246 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a8edac-a861-4592-a5e8-a2eb33a2a693 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:10.595869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9682d2-b29c-40ed-b91e-21390a1e20cc admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:10.608553 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc6588-f80d-4131-b057-b9511692f619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:10.617908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc6588-f80d-4131-b057-b9511692f619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:10.617908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc6588-f80d-4131-b057-b9511692f619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:10.617908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc6588-f80d-4131-b057-b9511692f619 admin admin] PATH_INFO: `/v3/projects/93b8558244294a49920234d44864e609` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:10.619366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc6588-f80d-4131-b057-b9511692f619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93b8558244294a49920234d44864e609)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:10.619726 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 838/1674] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:10 2022] PUT /identity/v3/projects/1dfdf2a73d98489bbf15b87ff309b922/users/b7de04d272d5496d88c115a4f2304ed4/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:10.621423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc6588-f80d-4131-b057-b9511692f619 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:10.621909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f307d-0ffa-452d-bc68-b56b9d5a5357 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:10.622956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f307d-0ffa-452d-bc68-b56b9d5a5357 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:10.623444 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f307d-0ffa-452d-bc68-b56b9d5a5357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:10.661457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ce1f307d-0ffa-452d-bc68-b56b9d5a5357 None None] MFA Rules not processed for user `b7de04d272d5496d88c115a4f2304ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:10.663246 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ce1f307d-0ffa-452d-bc68-b56b9d5a5357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:10.739148 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b0cc6588-f80d-4131-b057-b9511692f619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93b8558244294a49920234d44864e609', 'request_id': 'req-b0cc6588-f80d-4131-b057-b9511692f619'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:10.754434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b0cc6588-f80d-4131-b057-b9511692f619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93b8558244294a49920234d44864e609', 'request_id': 'req-b0cc6588-f80d-4131-b057-b9511692f619'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:10.767990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b0cc6588-f80d-4131-b057-b9511692f619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93b8558244294a49920234d44864e609', 'request_id': 'req-b0cc6588-f80d-4131-b057-b9511692f619'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:10.796732 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b0cc6588-f80d-4131-b057-b9511692f619 admin admin] The token cache is being invalidate because project 93b8558244294a49920234d44864e609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:10.797174 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b0cc6588-f80d-4131-b057-b9511692f619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:10.797909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b0cc6588-f80d-4131-b057-b9511692f619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:10.802495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 837/1675] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:10 2022] DELETE /identity/v3/projects/93b8558244294a49920234d44864e609 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:24:10.804854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074db65-8df0-4cee-83e1-5e1a34e0f70c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:10.805033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074db65-8df0-4cee-83e1-5e1a34e0f70c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:10.805363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074db65-8df0-4cee-83e1-5e1a34e0f70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:10.859978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b074db65-8df0-4cee-83e1-5e1a34e0f70c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:10.860152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 839/1676] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:10.866004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b074db65-8df0-4cee-83e1-5e1a34e0f70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:10.874204 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f307d-0ffa-452d-bc68-b56b9d5a5357 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:10.875604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ce1f307d-0ffa-452d-bc68-b56b9d5a5357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:10.882718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:10.950680 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:11.023732 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 838/1677] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:11.030733 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f307d-0ffa-452d-bc68-b56b9d5a5357 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:11.032860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6d219-7979-4f8c-a8a7-8902e605943c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:11.033263 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6d219-7979-4f8c-a8a7-8902e605943c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:11.033617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a6d219-7979-4f8c-a8a7-8902e605943c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:11.067836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-11a6d219-7979-4f8c-a8a7-8902e605943c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:11.068987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-11a6d219-7979-4f8c-a8a7-8902e605943c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:11.087485 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 839/1678] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:11.088713 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6d219-7979-4f8c-a8a7-8902e605943c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:11.112654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a6d219-7979-4f8c-a8a7-8902e605943c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:11.125957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a54a8f-5165-4122-8aa6-112adafb2ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:11.126676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a54a8f-5165-4122-8aa6-112adafb2ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:11.126961 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a54a8f-5165-4122-8aa6-112adafb2ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:11.127237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a54a8f-5165-4122-8aa6-112adafb2ca3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:11.130781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a54a8f-5165-4122-8aa6-112adafb2ca3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:11.132137 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a54a8f-5165-4122-8aa6-112adafb2ca3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:11.137263 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a54a8f-5165-4122-8aa6-112adafb2ca3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:11.160095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068290a4-7d24-403a-ba69-bc97472118a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:11.160727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068290a4-7d24-403a-ba69-bc97472118a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:11.161008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068290a4-7d24-403a-ba69-bc97472118a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:11.161788 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068290a4-7d24-403a-ba69-bc97472118a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:11.174160 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 840/1679] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:11.184007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a54a8f-5165-4122-8aa6-112adafb2ca3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:11.198049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a54a8f-5165-4122-8aa6-112adafb2ca3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:11.214665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b60b0-017a-412f-af85-574d579947b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:11.215295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b60b0-017a-412f-af85-574d579947b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:11.215574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b60b0-017a-412f-af85-574d579947b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:11.216420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b60b0-017a-412f-af85-574d579947b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:11.217074 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b60b0-017a-412f-af85-574d579947b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:11.249862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b60b0-017a-412f-af85-574d579947b5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:11.324736 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 841/1680] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:11 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:11.326063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b60b0-017a-412f-af85-574d579947b5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:11.363756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068290a4-7d24-403a-ba69-bc97472118a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:11.365430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068290a4-7d24-403a-ba69-bc97472118a9 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:11.414414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 840/1681] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:11.496657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b60b0-017a-412f-af85-574d579947b5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:11.529154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3cdad-2bf3-45d5-8509-94c29992c1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:11.530265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3cdad-2bf3-45d5-8509-94c29992c1da admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:11.530717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3cdad-2bf3-45d5-8509-94c29992c1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:11.531279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3cdad-2bf3-45d5-8509-94c29992c1da admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:11.533032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3cdad-2bf3-45d5-8509-94c29992c1da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:11.536439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3cdad-2bf3-45d5-8509-94c29992c1da admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:11.592699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 842/1682] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:11 2022] POST /identity/v3/users => generated 283 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:11.593659 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068290a4-7d24-403a-ba69-bc97472118a9 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:11.612510 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068290a4-7d24-403a-ba69-bc97472118a9 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:11.633146 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e93de2-0801-44dd-bf1b-5a522530d520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:11.633978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e93de2-0801-44dd-bf1b-5a522530d520 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:11.634478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e93de2-0801-44dd-bf1b-5a522530d520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:11.634908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e93de2-0801-44dd-bf1b-5a522530d520 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:11.635710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e93de2-0801-44dd-bf1b-5a522530d520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:11.637489 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e93de2-0801-44dd-bf1b-5a522530d520 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:11.660520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 841/1683] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:24:11.664918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3cdad-2bf3-45d5-8509-94c29992c1da admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:11.682340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3cdad-2bf3-45d5-8509-94c29992c1da admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:11.698200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0068692-9811-4f64-8bf6-239ea086a12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:11.698757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0068692-9811-4f64-8bf6-239ea086a12b admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:11.699041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0068692-9811-4f64-8bf6-239ea086a12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:11.699471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0068692-9811-4f64-8bf6-239ea086a12b admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:11.701901 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0068692-9811-4f64-8bf6-239ea086a12b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:11.708632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0068692-9811-4f64-8bf6-239ea086a12b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:11.735483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 843/1684] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:11.739713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e93de2-0801-44dd-bf1b-5a522530d520 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:11.759059 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e93de2-0801-44dd-bf1b-5a522530d520 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:11.773902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c6bf-5536-43f6-b4eb-0d8b7cb56891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:11.775283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c6bf-5536-43f6-b4eb-0d8b7cb56891 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:11.775943 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c6bf-5536-43f6-b4eb-0d8b7cb56891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:11.776465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235c6bf-5536-43f6-b4eb-0d8b7cb56891 admin admin] PATH_INFO: `/v3/projects/aec32d84a6fa47a3bc078bb3667a2884/users/439cfa5483344ad390e45fdc70f38c19/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:11.796795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235c6bf-5536-43f6-b4eb-0d8b7cb56891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aec32d84a6fa47a3bc078bb3667a2884, user_id=439cfa5483344ad390e45fdc70f38c19, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:11.800405 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235c6bf-5536-43f6-b4eb-0d8b7cb56891 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:11.827763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 842/1685] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:11 2022] PUT /identity/v3/projects/aec32d84a6fa47a3bc078bb3667a2884/users/439cfa5483344ad390e45fdc70f38c19/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:24:11.844338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5b34d-5b1b-4e72-8127-e80ec5e648cd None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:11.844894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5b34d-5b1b-4e72-8127-e80ec5e648cd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:11.844894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5b34d-5b1b-4e72-8127-e80ec5e648cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:11.913021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-42a5b34d-5b1b-4e72-8127-e80ec5e648cd None None] MFA Rules not processed for user `439cfa5483344ad390e45fdc70f38c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:11.915479 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-42a5b34d-5b1b-4e72-8127-e80ec5e648cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:12.078570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c6bf-5536-43f6-b4eb-0d8b7cb56891 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:12.098598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 844/1686] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:12.191388 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:12.265316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235c6bf-5536-43f6-b4eb-0d8b7cb56891 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:12.273351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91842c-c46d-4297-9ba9-c3d9a79d6e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:12.274059 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91842c-c46d-4297-9ba9-c3d9a79d6e37 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:12.274452 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91842c-c46d-4297-9ba9-c3d9a79d6e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:12.274793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91842c-c46d-4297-9ba9-c3d9a79d6e37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:12.275671 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91842c-c46d-4297-9ba9-c3d9a79d6e37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:12.277972 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91842c-c46d-4297-9ba9-c3d9a79d6e37 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:12.370457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 843/1687] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:12 2022] POST /identity/v3/projects => generated 365 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:12.374392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5b34d-5b1b-4e72-8127-e80ec5e648cd None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:12.381373 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:12.402909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5b34d-5b1b-4e72-8127-e80ec5e648cd None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:12.414461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91842c-c46d-4297-9ba9-c3d9a79d6e37 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:12.416636 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5f91842c-c46d-4297-9ba9-c3d9a79d6e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:12.570950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7553b0c-88e6-42ad-9d7a-51a5c4fb22cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:12.572483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7553b0c-88e6-42ad-9d7a-51a5c4fb22cf admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:12.575217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7553b0c-88e6-42ad-9d7a-51a5c4fb22cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:12.575637 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7553b0c-88e6-42ad-9d7a-51a5c4fb22cf admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:12.578560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7553b0c-88e6-42ad-9d7a-51a5c4fb22cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:12.581542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7553b0c-88e6-42ad-9d7a-51a5c4fb22cf admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:12.630264 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 845/1688] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:12 2022] POST /identity/v3/users => generated 289 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:12.631485 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7553b0c-88e6-42ad-9d7a-51a5c4fb22cf admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:12.633687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91842c-c46d-4297-9ba9-c3d9a79d6e37 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:12.654661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7553b0c-88e6-42ad-9d7a-51a5c4fb22cf admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:12.657068 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419cb94-0aed-4fe2-b6de-aac6b2240996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:12.657732 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419cb94-0aed-4fe2-b6de-aac6b2240996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:12.658008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419cb94-0aed-4fe2-b6de-aac6b2240996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:12.659500 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419cb94-0aed-4fe2-b6de-aac6b2240996 admin admin] PATH_INFO: `/v3/users/92e7560512b342d8a8e5330bc27d463f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:12.662096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419cb94-0aed-4fe2-b6de-aac6b2240996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92e7560512b342d8a8e5330bc27d463f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:12.664168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419cb94-0aed-4fe2-b6de-aac6b2240996 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:12.667393 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ecb5e-edde-48ac-9baa-e63f9b611f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:12.669658 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ecb5e-edde-48ac-9baa-e63f9b611f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:12.670026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ecb5e-edde-48ac-9baa-e63f9b611f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:12.670400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ecb5e-edde-48ac-9baa-e63f9b611f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:12.672296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ecb5e-edde-48ac-9baa-e63f9b611f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:12.680335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ecb5e-edde-48ac-9baa-e63f9b611f5b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:12.708038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 846/1689] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:12.709599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ecb5e-edde-48ac-9baa-e63f9b611f5b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:12.734599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ecb5e-edde-48ac-9baa-e63f9b611f5b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:12.746587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc93e2e-cfa2-4bc6-b974-ae8b18814ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:12.750647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc93e2e-cfa2-4bc6-b974-ae8b18814ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:12.751087 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc93e2e-cfa2-4bc6-b974-ae8b18814ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:12.751558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc93e2e-cfa2-4bc6-b974-ae8b18814ed3 admin admin] PATH_INFO: `/v3/projects/f28db939503d4fe2b7969a7dee575ed3/users/d950b6f57850439881bab3e0a5e99b99/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:12.787512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c419cb94-0aed-4fe2-b6de-aac6b2240996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92e7560512b342d8a8e5330bc27d463f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:12.787853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc93e2e-cfa2-4bc6-b974-ae8b18814ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f28db939503d4fe2b7969a7dee575ed3, user_id=d950b6f57850439881bab3e0a5e99b99, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:12.789764 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c419cb94-0aed-4fe2-b6de-aac6b2240996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92e7560512b342d8a8e5330bc27d463f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:12.793529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc93e2e-cfa2-4bc6-b974-ae8b18814ed3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:12.806716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c419cb94-0aed-4fe2-b6de-aac6b2240996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92e7560512b342d8a8e5330bc27d463f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:12.824698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 847/1690] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:12 2022] PUT /identity/v3/projects/f28db939503d4fe2b7969a7dee575ed3/users/d950b6f57850439881bab3e0a5e99b99/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:12.825404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c419cb94-0aed-4fe2-b6de-aac6b2240996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92e7560512b342d8a8e5330bc27d463f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:12.827086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc93e2e-cfa2-4bc6-b974-ae8b18814ed3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:12.831202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-bfc93e2e-cfa2-4bc6-b974-ae8b18814ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:12.861237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c419cb94-0aed-4fe2-b6de-aac6b2240996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92e7560512b342d8a8e5330bc27d463f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:12.878322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 844/1691] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:12 2022] DELETE /identity/v3/users/92e7560512b342d8a8e5330bc27d463f => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:12.882801 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612be82f-6194-4f81-b63c-9db5236ff0f2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:12.882801 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612be82f-6194-4f81-b63c-9db5236ff0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:12.883391 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612be82f-6194-4f81-b63c-9db5236ff0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:12.910596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-612be82f-6194-4f81-b63c-9db5236ff0f2 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:12.912230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-612be82f-6194-4f81-b63c-9db5236ff0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:12.960815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:13.005924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 845/1692] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:13.014619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612be82f-6194-4f81-b63c-9db5236ff0f2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:13.016443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-612be82f-6194-4f81-b63c-9db5236ff0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:13.044201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc93e2e-cfa2-4bc6-b974-ae8b18814ed3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:13.061222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fefd85-40bb-4610-bc5b-7807144df331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:13.061863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fefd85-40bb-4610-bc5b-7807144df331 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:13.062303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fefd85-40bb-4610-bc5b-7807144df331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:13.062586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fefd85-40bb-4610-bc5b-7807144df331 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:13.063253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fefd85-40bb-4610-bc5b-7807144df331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:13.064885 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fefd85-40bb-4610-bc5b-7807144df331 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:13.091903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 848/1693] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:13.093242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fefd85-40bb-4610-bc5b-7807144df331 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:13.117423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fefd85-40bb-4610-bc5b-7807144df331 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:13.137502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734dca5-baed-4b75-94d4-073f56e9500e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:13.139863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734dca5-baed-4b75-94d4-073f56e9500e admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:13.140148 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734dca5-baed-4b75-94d4-073f56e9500e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:13.140424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734dca5-baed-4b75-94d4-073f56e9500e admin admin] PATH_INFO: `/v3/projects/f28db939503d4fe2b7969a7dee575ed3/users/d950b6f57850439881bab3e0a5e99b99/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:13.146575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734dca5-baed-4b75-94d4-073f56e9500e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f28db939503d4fe2b7969a7dee575ed3, user_id=d950b6f57850439881bab3e0a5e99b99, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:13.148390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0734dca5-baed-4b75-94d4-073f56e9500e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:13.186467 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 849/1694] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:13 2022] PUT /identity/v3/projects/f28db939503d4fe2b7969a7dee575ed3/users/d950b6f57850439881bab3e0a5e99b99/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:13.188251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfa27f-9717-49ef-b75a-a54dc61c0fc4 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:13.188538 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfa27f-9717-49ef-b75a-a54dc61c0fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:13.189090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfa27f-9717-49ef-b75a-a54dc61c0fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:13.219782 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612be82f-6194-4f81-b63c-9db5236ff0f2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:13.267575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-46bfa27f-9717-49ef-b75a-a54dc61c0fc4 None None] MFA Rules not processed for user `d950b6f57850439881bab3e0a5e99b99`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:13.269039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-46bfa27f-9717-49ef-b75a-a54dc61c0fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:13.336440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:13.411821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d774b49-b3de-4bb3-92e0-3cba47b4401c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:13.412733 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d774b49-b3de-4bb3-92e0-3cba47b4401c service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:13.413081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d774b49-b3de-4bb3-92e0-3cba47b4401c service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:13.413751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d774b49-b3de-4bb3-92e0-3cba47b4401c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:13.465211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 850/1695] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:13.500678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfa27f-9717-49ef-b75a-a54dc61c0fc4 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:13.502588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-46bfa27f-9717-49ef-b75a-a54dc61c0fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:13.581168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d774b49-b3de-4bb3-92e0-3cba47b4401c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:13.587610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d774b49-b3de-4bb3-92e0-3cba47b4401c service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:13.603350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 846/1696] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:24:13.638679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfa27f-9717-49ef-b75a-a54dc61c0fc4 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:13.666031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8da112-702a-49e7-8a48-7472090eaa68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:13.666805 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8da112-702a-49e7-8a48-7472090eaa68 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:13.667081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8da112-702a-49e7-8a48-7472090eaa68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:13.667352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8da112-702a-49e7-8a48-7472090eaa68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:13.709975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8da112-702a-49e7-8a48-7472090eaa68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:13.711626 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8da112-702a-49e7-8a48-7472090eaa68 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:13.773169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 851/1697] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:24:13.960424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d774b49-b3de-4bb3-92e0-3cba47b4401c service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:13.973378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d774b49-b3de-4bb3-92e0-3cba47b4401c service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:13.986800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff8a2b-bd2e-4e70-911d-554ea316b4d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:13.987433 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff8a2b-bd2e-4e70-911d-554ea316b4d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:13.988286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff8a2b-bd2e-4e70-911d-554ea316b4d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:13.988756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff8a2b-bd2e-4e70-911d-554ea316b4d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:13.993743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dbff8a2b-bd2e-4e70-911d-554ea316b4d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:14.129110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:14.165230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff8a2b-bd2e-4e70-911d-554ea316b4d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:14.166757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff8a2b-bd2e-4e70-911d-554ea316b4d7 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:14.197888 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 847/1698] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:14.561683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8da112-702a-49e7-8a48-7472090eaa68 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:14.563641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cd8da112-702a-49e7-8a48-7472090eaa68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:14.600977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8da112-702a-49e7-8a48-7472090eaa68 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:14.618658 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:14.619316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:14.619942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:14.619942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0 admin admin] PATH_INFO: `/v3/projects/31f037b8adec484fa2e6756ec0ced4d2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:14.621809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31f037b8adec484fa2e6756ec0ced4d2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:14.630991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:14.769516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31f037b8adec484fa2e6756ec0ced4d2', 'request_id': 'req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:14.796151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31f037b8adec484fa2e6756ec0ced4d2', 'request_id': 'req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:14.807675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31f037b8adec484fa2e6756ec0ced4d2', 'request_id': 'req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:14.847498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0 admin admin] The token cache is being invalidate because project 31f037b8adec484fa2e6756ec0ced4d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:14.847883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:14.850439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eef162c3-bd0f-4a8e-9c2e-82a7c8a752f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:14.857872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 852/1699] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:14 2022] DELETE /identity/v3/projects/31f037b8adec484fa2e6756ec0ced4d2 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:14.861963 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff8a2b-bd2e-4e70-911d-554ea316b4d7 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:14.879998 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff8a2b-bd2e-4e70-911d-554ea316b4d7 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:15.050631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbad117-f294-46a7-8eac-50b689c10a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:15.052250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbad117-f294-46a7-8eac-50b689c10a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:15.053941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbad117-f294-46a7-8eac-50b689c10a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:15.054430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbad117-f294-46a7-8eac-50b689c10a87 admin admin] PATH_INFO: `/v3/users/b7de04d272d5496d88c115a4f2304ed4` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:15.057976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbad117-f294-46a7-8eac-50b689c10a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7de04d272d5496d88c115a4f2304ed4)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:15.060861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbad117-f294-46a7-8eac-50b689c10a87 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:15.151360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9bbad117-f294-46a7-8eac-50b689c10a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7de04d272d5496d88c115a4f2304ed4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:15.152244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9bbad117-f294-46a7-8eac-50b689c10a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7de04d272d5496d88c115a4f2304ed4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:15.163155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9bbad117-f294-46a7-8eac-50b689c10a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7de04d272d5496d88c115a4f2304ed4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:15.181636 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9bbad117-f294-46a7-8eac-50b689c10a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7de04d272d5496d88c115a4f2304ed4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:15.208426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9bbad117-f294-46a7-8eac-50b689c10a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7de04d272d5496d88c115a4f2304ed4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:15.220836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 848/1700] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:14 2022] DELETE /identity/v3/users/b7de04d272d5496d88c115a4f2304ed4 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:15.300963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9412d5-1c03-4d12-be1e-b245c8165ae4 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:15.302555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9412d5-1c03-4d12-be1e-b245c8165ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:15.302920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9412d5-1c03-4d12-be1e-b245c8165ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:15.399459 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1b9412d5-1c03-4d12-be1e-b245c8165ae4 None None] MFA Rules not processed for user `d23a498bde00483c8fdd68dcde975e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:15.400812 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1b9412d5-1c03-4d12-be1e-b245c8165ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:15.520884 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:15.582942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 853/1701] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:15.594037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fc6fb-e930-4534-8192-302f06b6efb3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:15.594037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fc6fb-e930-4534-8192-302f06b6efb3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:15.594037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fc6fb-e930-4534-8192-302f06b6efb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:15.634878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b58fc6fb-e930-4534-8192-302f06b6efb3 None None] MFA Rules not processed for user `d23a498bde00483c8fdd68dcde975e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:15.636463 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b58fc6fb-e930-4534-8192-302f06b6efb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:15.641072 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9412d5-1c03-4d12-be1e-b245c8165ae4 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:15.642620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1b9412d5-1c03-4d12-be1e-b245c8165ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:15.658969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 849/1702] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:15.670250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fc6fb-e930-4534-8192-302f06b6efb3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:15.672045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b58fc6fb-e930-4534-8192-302f06b6efb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:15.783754 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9412d5-1c03-4d12-be1e-b245c8165ae4 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:15.794383 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5a2fd-c6c4-4164-ae58-60fc7d1b69c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:15.795310 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5a2fd-c6c4-4164-ae58-60fc7d1b69c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:15.795593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5a2fd-c6c4-4164-ae58-60fc7d1b69c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:15.795903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5a2fd-c6c4-4164-ae58-60fc7d1b69c7 admin admin] PATH_INFO: `/v3/users/1b0bf21eb066497db842c72c63a95f4a` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:15.797086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5a2fd-c6c4-4164-ae58-60fc7d1b69c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b0bf21eb066497db842c72c63a95f4a)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:15.798529 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:15.799535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5a2fd-c6c4-4164-ae58-60fc7d1b69c7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:15.844869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fc6fb-e930-4534-8192-302f06b6efb3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:15.862216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214fad4-4bd3-4947-97bc-b30774861765 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:15.863757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214fad4-4bd3-4947-97bc-b30774861765 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:15.864251 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214fad4-4bd3-4947-97bc-b30774861765 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:15.864632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214fad4-4bd3-4947-97bc-b30774861765 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:15.887745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214fad4-4bd3-4947-97bc-b30774861765 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:15.890405 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214fad4-4bd3-4947-97bc-b30774861765 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:15.920491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 850/1703] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:24:15.924370 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69916f45-61bd-45ba-92f9-0a5e53b72a3c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:15.924842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69916f45-61bd-45ba-92f9-0a5e53b72a3c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:15.925934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b4f5a2fd-c6c4-4164-ae58-60fc7d1b69c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b0bf21eb066497db842c72c63a95f4a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:15.926048 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69916f45-61bd-45ba-92f9-0a5e53b72a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:15.937266 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b4f5a2fd-c6c4-4164-ae58-60fc7d1b69c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b0bf21eb066497db842c72c63a95f4a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:15.938680 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b4f5a2fd-c6c4-4164-ae58-60fc7d1b69c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b0bf21eb066497db842c72c63a95f4a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:15.954357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b4f5a2fd-c6c4-4164-ae58-60fc7d1b69c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b0bf21eb066497db842c72c63a95f4a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:15.978804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b4f5a2fd-c6c4-4164-ae58-60fc7d1b69c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b0bf21eb066497db842c72c63a95f4a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:15.988364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 854/1704] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:15 2022] DELETE /identity/v3/users/1b0bf21eb066497db842c72c63a95f4a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:15.998655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e0b0b-2206-4526-bbb8-db638beea79d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:15.998837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e0b0b-2206-4526-bbb8-db638beea79d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:16.000752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e0b0b-2206-4526-bbb8-db638beea79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:16.030773 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-69916f45-61bd-45ba-92f9-0a5e53b72a3c None None] MFA Rules not processed for user `1dde550901f04ea695b85db4e41d9628`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:16.033756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-69916f45-61bd-45ba-92f9-0a5e53b72a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:16.063709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-633e0b0b-2206-4526-bbb8-db638beea79d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:16.065752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-633e0b0b-2206-4526-bbb8-db638beea79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:16.134418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:16.159468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 855/1705] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:16.162060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e0b0b-2206-4526-bbb8-db638beea79d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:16.167223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-633e0b0b-2206-4526-bbb8-db638beea79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:16.192582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e0b0b-2206-4526-bbb8-db638beea79d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:16.206161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 851/1706] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:16.206161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69916f45-61bd-45ba-92f9-0a5e53b72a3c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:16.209490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:16.209644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-69916f45-61bd-45ba-92f9-0a5e53b72a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:16.209727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:16.209727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:16.209727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] PATH_INFO: `/v3/projects/1dfdf2a73d98489bbf15b87ff309b922` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:16.210388 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dfdf2a73d98489bbf15b87ff309b922)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:16.212557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:16.334019 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dfdf2a73d98489bbf15b87ff309b922', 'request_id': 'req-6311a07a-2552-402e-825e-5d21c01b7d5d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:16.342024 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dfdf2a73d98489bbf15b87ff309b922', 'request_id': 'req-6311a07a-2552-402e-825e-5d21c01b7d5d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:16.350336 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dfdf2a73d98489bbf15b87ff309b922', 'request_id': 'req-6311a07a-2552-402e-825e-5d21c01b7d5d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:16.364275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69916f45-61bd-45ba-92f9-0a5e53b72a3c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:16.379498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] The token cache is being invalidate because project 1dfdf2a73d98489bbf15b87ff309b922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:16.379801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:16.381362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:16.389924 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 856/1707] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:16 2022] DELETE /identity/v3/projects/1dfdf2a73d98489bbf15b87ff309b922 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:16.391736 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:16.398942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:16.465486 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb767b3-a0cf-4b60-bfbb-06e949d5807e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:16.466510 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb767b3-a0cf-4b60-bfbb-06e949d5807e service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:16.467165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb767b3-a0cf-4b60-bfbb-06e949d5807e service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:16.467647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb767b3-a0cf-4b60-bfbb-06e949d5807e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:16.469878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cbb767b3-a0cf-4b60-bfbb-06e949d5807e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:16.572171 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311a07a-2552-402e-825e-5d21c01b7d5d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:16.575598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:16.585728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9852d-9f67-47da-b064-35a28f91a00b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:16.586475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9852d-9f67-47da-b064-35a28f91a00b service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:16.586761 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9852d-9f67-47da-b064-35a28f91a00b service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:16.587035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9852d-9f67-47da-b064-35a28f91a00b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:16.590498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-77b9852d-9f67-47da-b064-35a28f91a00b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:16.630444 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb767b3-a0cf-4b60-bfbb-06e949d5807e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:16.632113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb767b3-a0cf-4b60-bfbb-06e949d5807e service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:16.651088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 852/1708] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:16.652797 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df2257-c2bf-44c7-89a8-2052e0e0ea62 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:16.653385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df2257-c2bf-44c7-89a8-2052e0e0ea62 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:16.653835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df2257-c2bf-44c7-89a8-2052e0e0ea62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:16.699605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a0df2257-c2bf-44c7-89a8-2052e0e0ea62 None None] MFA Rules not processed for user `e48c146c0bba4583a92d157e2eed7f99`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:16.702101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a0df2257-c2bf-44c7-89a8-2052e0e0ea62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:16.828763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9852d-9f67-47da-b064-35a28f91a00b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:16.830952 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9852d-9f67-47da-b064-35a28f91a00b service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:16.849401 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 853/1709] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:16.850605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df2257-c2bf-44c7-89a8-2052e0e0ea62 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:16.851694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 857/1710] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:16.854320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd4740-5528-4326-b1f0-051766c91296 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:16.855949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd4740-5528-4326-b1f0-051766c91296 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:16.855949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd4740-5528-4326-b1f0-051766c91296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:16.876608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a0df2257-c2bf-44c7-89a8-2052e0e0ea62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:16.908030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c1cd4740-5528-4326-b1f0-051766c91296 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:16.910332 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c1cd4740-5528-4326-b1f0-051766c91296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:16.911521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df2257-c2bf-44c7-89a8-2052e0e0ea62 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:16.923293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9764f0-757c-4e01-9c41-f553d3da5dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:16.925758 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9764f0-757c-4e01-9c41-f553d3da5dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:16.926133 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9764f0-757c-4e01-9c41-f553d3da5dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:16.926619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9764f0-757c-4e01-9c41-f553d3da5dc6 admin admin] PATH_INFO: `/v3/users/9300e254fc1549bc9569d4de78718afb` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:16.930049 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9764f0-757c-4e01-9c41-f553d3da5dc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9300e254fc1549bc9569d4de78718afb)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:16.934160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9764f0-757c-4e01-9c41-f553d3da5dc6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:16.939319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 858/1711] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:16.945136 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f94f2f-c729-4f39-9754-2155585e7378 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:16.945668 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f94f2f-c729-4f39-9754-2155585e7378 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:16.945898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f94f2f-c729-4f39-9754-2155585e7378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:17.003176 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-61f94f2f-c729-4f39-9754-2155585e7378 None None] MFA Rules not processed for user `e48c146c0bba4583a92d157e2eed7f99`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:17.007431 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-61f94f2f-c729-4f39-9754-2155585e7378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:17.025396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 859/1712] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:17.029722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0069ac-95e4-436a-acaa-8ad5f99d123d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:17.029722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0069ac-95e4-436a-acaa-8ad5f99d123d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:17.029722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0069ac-95e4-436a-acaa-8ad5f99d123d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:17.041348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 860/1713] 10.222.0.55 () {54 vars in 912 bytes} [Tue Sep 27 17:24:17 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:24:17.044587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488b6f4-9990-4c3d-8574-32d99e8193f9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:17.050602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488b6f4-9990-4c3d-8574-32d99e8193f9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:17.050889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488b6f4-9990-4c3d-8574-32d99e8193f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:17.077474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:17.090128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe9764f0-757c-4e01-9c41-f553d3da5dc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9300e254fc1549bc9569d4de78718afb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:17.091985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe9764f0-757c-4e01-9c41-f553d3da5dc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9300e254fc1549bc9569d4de78718afb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:17.103243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0488b6f4-9990-4c3d-8574-32d99e8193f9 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:17.104973 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe9764f0-757c-4e01-9c41-f553d3da5dc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9300e254fc1549bc9569d4de78718afb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:17.106596 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0488b6f4-9990-4c3d-8574-32d99e8193f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:17.118542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 861/1714] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:17.118542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3bf49-a938-41b1-8d70-cb94dc30cff9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:17.118542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3bf49-a938-41b1-8d70-cb94dc30cff9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:17.118542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3bf49-a938-41b1-8d70-cb94dc30cff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:17.127332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe9764f0-757c-4e01-9c41-f553d3da5dc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9300e254fc1549bc9569d4de78718afb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:17.161728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe9764f0-757c-4e01-9c41-f553d3da5dc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9300e254fc1549bc9569d4de78718afb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:17.173796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 854/1715] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:16 2022] DELETE /identity/v3/users/9300e254fc1549bc9569d4de78718afb => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:17.174488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9764f0-757c-4e01-9c41-f553d3da5dc6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:17.220219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b1f3bf49-a938-41b1-8d70-cb94dc30cff9 None None] MFA Rules not processed for user `5d906452199844149966c9c86db4684e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:17.221508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b1f3bf49-a938-41b1-8d70-cb94dc30cff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:17.266749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:17.342586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 862/1716] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:24:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:17.342586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c98a16-01e1-4aef-aa6e-2a70c086587b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:17.342586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c98a16-01e1-4aef-aa6e-2a70c086587b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:17.342586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c98a16-01e1-4aef-aa6e-2a70c086587b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:17.359461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9764f0-757c-4e01-9c41-f553d3da5dc6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:17.367609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-82c98a16-01e1-4aef-aa6e-2a70c086587b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:17.368857 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-82c98a16-01e1-4aef-aa6e-2a70c086587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:17.371271 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9758c7-7371-4fbd-9c0f-e982ecc6285d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:17.372475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9758c7-7371-4fbd-9c0f-e982ecc6285d admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:17.372757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9758c7-7371-4fbd-9c0f-e982ecc6285d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:17.373031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9758c7-7371-4fbd-9c0f-e982ecc6285d admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:17.373949 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9758c7-7371-4fbd-9c0f-e982ecc6285d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:17.374446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9758c7-7371-4fbd-9c0f-e982ecc6285d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:17.376229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9758c7-7371-4fbd-9c0f-e982ecc6285d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:17.389138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 863/1717] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:17.390544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c98a16-01e1-4aef-aa6e-2a70c086587b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:17.395500 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-82c98a16-01e1-4aef-aa6e-2a70c086587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:17.414348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 855/1718] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:17.415593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9758c7-7371-4fbd-9c0f-e982ecc6285d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:17.417483 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1a9758c7-7371-4fbd-9c0f-e982ecc6285d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:17.433955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9758c7-7371-4fbd-9c0f-e982ecc6285d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:17.440686 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:17.441362 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:17.441639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:17.441912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] PATH_INFO: `/v3/projects/b475d9ed19fb470aa81d133feadb658a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:17.443097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b475d9ed19fb470aa81d133feadb658a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:17.445049 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:17.524965 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c98a16-01e1-4aef-aa6e-2a70c086587b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:17.547406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dacb24-13e0-4f3e-a96b-ead09e526eca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:17.549352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dacb24-13e0-4f3e-a96b-ead09e526eca service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:17.550634 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dacb24-13e0-4f3e-a96b-ead09e526eca service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:17.551654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dacb24-13e0-4f3e-a96b-ead09e526eca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:17.555597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-41dacb24-13e0-4f3e-a96b-ead09e526eca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:17.571868 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b475d9ed19fb470aa81d133feadb658a', 'request_id': 'req-7a1c5762-838b-4db8-a5f8-00107ed57881'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:17.585765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b475d9ed19fb470aa81d133feadb658a', 'request_id': 'req-7a1c5762-838b-4db8-a5f8-00107ed57881'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:17.603414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b475d9ed19fb470aa81d133feadb658a', 'request_id': 'req-7a1c5762-838b-4db8-a5f8-00107ed57881'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:17.632199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] The token cache is being invalidate because project b475d9ed19fb470aa81d133feadb658a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:17.634566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:17.636255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:17.644467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 856/1719] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:17 2022] DELETE /identity/v3/projects/b475d9ed19fb470aa81d133feadb658a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:17.645712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:17.649265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:17.719560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dacb24-13e0-4f3e-a96b-ead09e526eca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:17.723284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dacb24-13e0-4f3e-a96b-ead09e526eca service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:17.749980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 864/1720] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:17.751918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dacb24-13e0-4f3e-a96b-ead09e526eca service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:17.755977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-41dacb24-13e0-4f3e-a96b-ead09e526eca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:17.789833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dacb24-13e0-4f3e-a96b-ead09e526eca service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:17.799752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e0f28-6f73-49f5-86f7-58bf664443c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:17.800391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e0f28-6f73-49f5-86f7-58bf664443c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:17.800715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e0f28-6f73-49f5-86f7-58bf664443c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:17.800987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e0f28-6f73-49f5-86f7-58bf664443c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:17.802175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c5762-838b-4db8-a5f8-00107ed57881 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:17.803036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8b2e0f28-6f73-49f5-86f7-58bf664443c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:17.806177 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c85188-665a-4eaa-b41f-015c32f7b990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:17.807015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c85188-665a-4eaa-b41f-015c32f7b990 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:17.807647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c85188-665a-4eaa-b41f-015c32f7b990 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:17.808022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c85188-665a-4eaa-b41f-015c32f7b990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:17.810958 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-66c85188-665a-4eaa-b41f-015c32f7b990 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:17.822035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c85188-665a-4eaa-b41f-015c32f7b990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:17.839989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c85188-665a-4eaa-b41f-015c32f7b990 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:17.883616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:17.891097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 857/1721] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:17.892315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c85188-665a-4eaa-b41f-015c32f7b990 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:17.896023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-66c85188-665a-4eaa-b41f-015c32f7b990 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:17.987781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e0f28-6f73-49f5-86f7-58bf664443c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:17.989347 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e0f28-6f73-49f5-86f7-58bf664443c2 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:18.013664 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 865/1722] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:18.015001 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79886751-8574-4984-a2f7-6c1a0d722998 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:18.015285 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79886751-8574-4984-a2f7-6c1a0d722998 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:18.015560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79886751-8574-4984-a2f7-6c1a0d722998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:18.067229 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-79886751-8574-4984-a2f7-6c1a0d722998 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:18.068484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-79886751-8574-4984-a2f7-6c1a0d722998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:18.100683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 866/1723] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:18.109766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51521154-bb69-447f-beb0-aa9797cb4e87 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:18.110148 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51521154-bb69-447f-beb0-aa9797cb4e87 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:18.110486 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51521154-bb69-447f-beb0-aa9797cb4e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:18.152432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:18.153980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-51521154-bb69-447f-beb0-aa9797cb4e87 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:18.155624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-51521154-bb69-447f-beb0-aa9797cb4e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:18.163798 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c85188-665a-4eaa-b41f-015c32f7b990 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:18.173220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa7789-c0e7-4c7c-8e64-821ce2b9b3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:18.174148 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa7789-c0e7-4c7c-8e64-821ce2b9b3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:18.174910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa7789-c0e7-4c7c-8e64-821ce2b9b3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:18.175966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa7789-c0e7-4c7c-8e64-821ce2b9b3b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:18.181774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa7789-c0e7-4c7c-8e64-821ce2b9b3b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:18.185823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 867/1724] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:24:18.185823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51521154-bb69-447f-beb0-aa9797cb4e87 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:18.188292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa7789-c0e7-4c7c-8e64-821ce2b9b3b8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:18.189972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-51521154-bb69-447f-beb0-aa9797cb4e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:18.221078 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51521154-bb69-447f-beb0-aa9797cb4e87 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:18.230758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:18.232114 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:18.237037 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:18.238769 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] PATH_INFO: `/v3/users/439cfa5483344ad390e45fdc70f38c19` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:18.242662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=439cfa5483344ad390e45fdc70f38c19)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:18.246208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:18.250823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 858/1725] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:17 2022] POST /identity/v3/projects => generated 375 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:18.252032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa7789-c0e7-4c7c-8e64-821ce2b9b3b8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:18.343636 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '439cfa5483344ad390e45fdc70f38c19'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:18.365473 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '439cfa5483344ad390e45fdc70f38c19'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:18.367247 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '439cfa5483344ad390e45fdc70f38c19'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:18.383115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '439cfa5483344ad390e45fdc70f38c19'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:18.410535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa7789-c0e7-4c7c-8e64-821ce2b9b3b8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:18.414378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '439cfa5483344ad390e45fdc70f38c19'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:18.424472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2186f5ba-d1f2-417c-b4dc-b09660b81b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:18.425344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186f5ba-d1f2-417c-b4dc-b09660b81b92 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:18.425756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186f5ba-d1f2-417c-b4dc-b09660b81b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:18.427524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2186f5ba-d1f2-417c-b4dc-b09660b81b92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:18.428317 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186f5ba-d1f2-417c-b4dc-b09660b81b92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:18.430651 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186f5ba-d1f2-417c-b4dc-b09660b81b92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:18.432215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 868/1726] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:18 2022] DELETE /identity/v3/users/439cfa5483344ad390e45fdc70f38c19 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:18.433809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:18.435719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2186f5ba-d1f2-417c-b4dc-b09660b81b92 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:18.438674 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:18.459686 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 859/1727] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:18.461943 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2a283-7f60-4b74-9049-c36ba6d5bc1c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:18.462459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2a283-7f60-4b74-9049-c36ba6d5bc1c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:18.462834 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2a283-7f60-4b74-9049-c36ba6d5bc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:18.493666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-28b2a283-7f60-4b74-9049-c36ba6d5bc1c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:18.497393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-28b2a283-7f60-4b74-9049-c36ba6d5bc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:18.586495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f2dbb4-4aec-4210-93fb-3c1e1d4b85e2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:18.598294 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ed44b-00a7-4720-b73c-7153126b0d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:18.600419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ed44b-00a7-4720-b73c-7153126b0d0c admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:18.601151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ed44b-00a7-4720-b73c-7153126b0d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:18.602420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ed44b-00a7-4720-b73c-7153126b0d0c admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:18.603739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ed44b-00a7-4720-b73c-7153126b0d0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:18.608460 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 860/1728] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:18.609447 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2a283-7f60-4b74-9049-c36ba6d5bc1c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:18.611875 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ed44b-00a7-4720-b73c-7153126b0d0c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:18.614859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-28b2a283-7f60-4b74-9049-c36ba6d5bc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:18.645455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2a283-7f60-4b74-9049-c36ba6d5bc1c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:18.658369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb4638-cb81-4f99-868c-92e9f65b1750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:18.658992 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb4638-cb81-4f99-868c-92e9f65b1750 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:18.659273 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb4638-cb81-4f99-868c-92e9f65b1750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:18.659749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb4638-cb81-4f99-868c-92e9f65b1750 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:18.661677 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb4638-cb81-4f99-868c-92e9f65b1750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:18.662173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 869/1729] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:18 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:18.663146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ed44b-00a7-4720-b73c-7153126b0d0c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:18.667828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ea5ed44b-00a7-4720-b73c-7153126b0d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:18.669421 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb4638-cb81-4f99-868c-92e9f65b1750 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:18.696597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:18.746475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 861/1730] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:18 2022] POST /identity/v3/projects => generated 365 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:18.747939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb4638-cb81-4f99-868c-92e9f65b1750 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:18.785028 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ed44b-00a7-4720-b73c-7153126b0d0c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:18.874386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:18.893221 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb4638-cb81-4f99-868c-92e9f65b1750 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:18.911102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9256ee-6bda-4d37-92b7-79e320fbe54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:18.913219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9256ee-6bda-4d37-92b7-79e320fbe54b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:18.913516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9256ee-6bda-4d37-92b7-79e320fbe54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:18.913794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9256ee-6bda-4d37-92b7-79e320fbe54b admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:18.917747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9256ee-6bda-4d37-92b7-79e320fbe54b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:18.921123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9256ee-6bda-4d37-92b7-79e320fbe54b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:18.957284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 862/1731] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:18.958276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9256ee-6bda-4d37-92b7-79e320fbe54b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:18.964937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483aef33-d0e5-47ab-9e2a-20c0b1a1ef92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:18.966116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483aef33-d0e5-47ab-9e2a-20c0b1a1ef92 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:18.966400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483aef33-d0e5-47ab-9e2a-20c0b1a1ef92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:18.966669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483aef33-d0e5-47ab-9e2a-20c0b1a1ef92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:18.982377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9256ee-6bda-4d37-92b7-79e320fbe54b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:18.983973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483aef33-d0e5-47ab-9e2a-20c0b1a1ef92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:18.986212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483aef33-d0e5-47ab-9e2a-20c0b1a1ef92 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:18.996307 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca80f6-8daa-4901-8e3d-3ad8d637893e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:18.997447 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca80f6-8daa-4901-8e3d-3ad8d637893e admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:18.997724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca80f6-8daa-4901-8e3d-3ad8d637893e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:18.997993 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca80f6-8daa-4901-8e3d-3ad8d637893e admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:18.998661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca80f6-8daa-4901-8e3d-3ad8d637893e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:19.000593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca80f6-8daa-4901-8e3d-3ad8d637893e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:19.009915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 870/1732] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:19.011720 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff7de0-35e0-44a2-9b26-e9b6e803ada4 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:19.012028 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff7de0-35e0-44a2-9b26-e9b6e803ada4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:19.012725 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff7de0-35e0-44a2-9b26-e9b6e803ada4 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:19.016640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 871/1733] 10.222.0.55 () {54 vars in 912 bytes} [Tue Sep 27 17:24:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:24:19.019166 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff7de0-35e0-44a2-9b26-e9b6e803ada4 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:19.052277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 863/1734] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:18 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:19.060144 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff7de0-35e0-44a2-9b26-e9b6e803ada4 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:19.070386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59e980-5cf1-403a-8af5-ea0f98ccb5c4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:19.070386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59e980-5cf1-403a-8af5-ea0f98ccb5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:19.070386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59e980-5cf1-403a-8af5-ea0f98ccb5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:19.081270 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6492f07-aa15-4890-b5e8-62afc1112c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:19.081886 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6492f07-aa15-4890-b5e8-62afc1112c53 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:19.082191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6492f07-aa15-4890-b5e8-62afc1112c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:19.082462 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6492f07-aa15-4890-b5e8-62afc1112c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:19.083109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6492f07-aa15-4890-b5e8-62afc1112c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:19.086692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6492f07-aa15-4890-b5e8-62afc1112c53 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:19.125015 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 872/1735] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:19.129507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8d59e980-5cf1-403a-8af5-ea0f98ccb5c4 None None] MFA Rules not processed for user `5d906452199844149966c9c86db4684e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:19.130784 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8d59e980-5cf1-403a-8af5-ea0f98ccb5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:19.132566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6492f07-aa15-4890-b5e8-62afc1112c53 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:19.157699 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 864/1736] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:24:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:19.160182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59e980-5cf1-403a-8af5-ea0f98ccb5c4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:19.162521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6492f07-aa15-4890-b5e8-62afc1112c53 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:19.181467 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218aef2-fd44-4002-8692-e508bdb8f0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:19.185410 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218aef2-fd44-4002-8692-e508bdb8f0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:19.185695 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218aef2-fd44-4002-8692-e508bdb8f0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:19.185964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218aef2-fd44-4002-8692-e508bdb8f0a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:19.186671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218aef2-fd44-4002-8692-e508bdb8f0a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:19.199048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3218aef2-fd44-4002-8692-e508bdb8f0a3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:19.201974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59e980-5cf1-403a-8af5-ea0f98ccb5c4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:19.220557 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ffab3-a562-411a-acf4-5b51f8e7e33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:19.227164 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ffab3-a562-411a-acf4-5b51f8e7e33d admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:19.227552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ffab3-a562-411a-acf4-5b51f8e7e33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:19.227837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ffab3-a562-411a-acf4-5b51f8e7e33d admin admin] PATH_INFO: `/v3/projects/61f5f14379f04984bdf40631daeac3d8/users/2f80c00777454d60a695008760076f6e/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:19.233800 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 873/1737] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:19.234862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218aef2-fd44-4002-8692-e508bdb8f0a3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:19.243146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3218aef2-fd44-4002-8692-e508bdb8f0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:19.280051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ffab3-a562-411a-acf4-5b51f8e7e33d admin admin] RBAC: Authorizing `identity:create_grant(project_id=61f5f14379f04984bdf40631daeac3d8, user_id=2f80c00777454d60a695008760076f6e, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:19.283950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ffab3-a562-411a-acf4-5b51f8e7e33d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:19.284504 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218aef2-fd44-4002-8692-e508bdb8f0a3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:19.291790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752bf71-cf95-46d8-a78a-8cf756e8f2ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:19.294131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752bf71-cf95-46d8-a78a-8cf756e8f2ed service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:19.294131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752bf71-cf95-46d8-a78a-8cf756e8f2ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:19.294362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752bf71-cf95-46d8-a78a-8cf756e8f2ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:19.315036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 865/1738] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:19 2022] PUT /identity/v3/projects/61f5f14379f04984bdf40631daeac3d8/users/2f80c00777454d60a695008760076f6e/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:19.316295 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ffab3-a562-411a-acf4-5b51f8e7e33d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:19.317041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752bf71-cf95-46d8-a78a-8cf756e8f2ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:19.332094 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752bf71-cf95-46d8-a78a-8cf756e8f2ed service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:19.360586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 874/1739] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:19.361867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff47a-9788-49f6-bde4-b029d91b20fd None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:19.362184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff47a-9788-49f6-bde4-b029d91b20fd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:19.362459 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff47a-9788-49f6-bde4-b029d91b20fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:19.437720 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e5eff47a-9788-49f6-bde4-b029d91b20fd None None] MFA Rules not processed for user `2f80c00777454d60a695008760076f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:19.439395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e5eff47a-9788-49f6-bde4-b029d91b20fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:19.460828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ffab3-a562-411a-acf4-5b51f8e7e33d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:19.490985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bdb2f-2166-4c2b-a31a-542f9bf185a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:19.492194 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bdb2f-2166-4c2b-a31a-542f9bf185a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:19.492878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bdb2f-2166-4c2b-a31a-542f9bf185a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:19.493357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841bdb2f-2166-4c2b-a31a-542f9bf185a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:19.494534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841bdb2f-2166-4c2b-a31a-542f9bf185a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:19.496536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841bdb2f-2166-4c2b-a31a-542f9bf185a7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:19.539781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 866/1740] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:19.542576 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bdb2f-2166-4c2b-a31a-542f9bf185a7 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:19.548569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-841bdb2f-2166-4c2b-a31a-542f9bf185a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:19.583953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 875/1741] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:19.584943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eff47a-9788-49f6-bde4-b029d91b20fd None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:19.598682 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841bdb2f-2166-4c2b-a31a-542f9bf185a7 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:19.609299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eff47a-9788-49f6-bde4-b029d91b20fd None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:19.621660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:19.623097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdae219-cb3e-42c2-9f9c-494d37a954a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:19.623236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:19.623414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:19.623632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] PATH_INFO: `/v3/users/766279c94408482a8438cb44bebadbc5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:19.623722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdae219-cb3e-42c2-9f9c-494d37a954a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:19.623875 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdae219-cb3e-42c2-9f9c-494d37a954a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:19.624340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdae219-cb3e-42c2-9f9c-494d37a954a3 admin admin] PATH_INFO: `/v3/projects/daea53c18fc1484ab977aaf49e46c204/users/bfaf4a051e0149a18851bdc11be1f122/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:19.626838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=766279c94408482a8438cb44bebadbc5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:19.629347 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:19.654313 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdae219-cb3e-42c2-9f9c-494d37a954a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daea53c18fc1484ab977aaf49e46c204, user_id=bfaf4a051e0149a18851bdc11be1f122, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:19.656999 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdae219-cb3e-42c2-9f9c-494d37a954a3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:19.701458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:19.710230 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 876/1742] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:19 2022] PUT /identity/v3/projects/daea53c18fc1484ab977aaf49e46c204/users/bfaf4a051e0149a18851bdc11be1f122/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:19.725062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdae219-cb3e-42c2-9f9c-494d37a954a3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:19.732314 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9bdae219-cb3e-42c2-9f9c-494d37a954a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:19.789821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '766279c94408482a8438cb44bebadbc5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:19.791536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '766279c94408482a8438cb44bebadbc5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:19.830513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '766279c94408482a8438cb44bebadbc5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:19.845220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '766279c94408482a8438cb44bebadbc5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:19.869283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '766279c94408482a8438cb44bebadbc5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:19.877303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 867/1743] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:19 2022] DELETE /identity/v3/users/766279c94408482a8438cb44bebadbc5 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:19.877928 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:19.881882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:19.906264 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:19.984403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdae219-cb3e-42c2-9f9c-494d37a954a3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:20.071585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68f3ce-3e35-4dc8-b2e4-26b8a0b6c439 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:20.083737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38b80d-e945-425c-9579-041c06442804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:20.085801 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38b80d-e945-425c-9579-041c06442804 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:20.086149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38b80d-e945-425c-9579-041c06442804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:20.086394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38b80d-e945-425c-9579-041c06442804 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:20.088140 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38b80d-e945-425c-9579-041c06442804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:20.096907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38b80d-e945-425c-9579-041c06442804 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:20.160021 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 868/1744] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:19 2022] POST /identity/v3/projects => generated 377 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:20.168905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee57491-d4c2-452b-8f74-1ae21b9dd430 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:20.169215 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee57491-d4c2-452b-8f74-1ae21b9dd430 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:20.169499 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee57491-d4c2-452b-8f74-1ae21b9dd430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:20.175695 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc7b1b-050f-41d5-b8b0-0829a4e0d796 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:20.176332 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc7b1b-050f-41d5-b8b0-0829a4e0d796 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:20.176608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc7b1b-050f-41d5-b8b0-0829a4e0d796 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:20.176877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc7b1b-050f-41d5-b8b0-0829a4e0d796 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:20.179442 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-19fc7b1b-050f-41d5-b8b0-0829a4e0d796 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:20.207354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc7b1b-050f-41d5-b8b0-0829a4e0d796 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:20.209169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc7b1b-050f-41d5-b8b0-0829a4e0d796 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:20.214431 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6ee57491-d4c2-452b-8f74-1ae21b9dd430 None None] MFA Rules not processed for user `bfaf4a051e0149a18851bdc11be1f122`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:20.216482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6ee57491-d4c2-452b-8f74-1ae21b9dd430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:20.236343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 877/1745] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:20.237663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2251b8-78bc-4ff2-a568-2bf00fdf6477 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:20.238106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2251b8-78bc-4ff2-a568-2bf00fdf6477 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:20.238462 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2251b8-78bc-4ff2-a568-2bf00fdf6477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:20.284937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ad2251b8-78bc-4ff2-a568-2bf00fdf6477 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:20.286197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ad2251b8-78bc-4ff2-a568-2bf00fdf6477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:20.493845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 869/1746] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:20.497733 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39734a1-7390-4169-8095-51d614751e75 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:20.499144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39734a1-7390-4169-8095-51d614751e75 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:20.499490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39734a1-7390-4169-8095-51d614751e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:20.517521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 878/1747] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:20.518533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2251b8-78bc-4ff2-a568-2bf00fdf6477 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:20.562402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2251b8-78bc-4ff2-a568-2bf00fdf6477 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:20.565208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b39734a1-7390-4169-8095-51d614751e75 None None] MFA Rules not processed for user `d23a498bde00483c8fdd68dcde975e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:20.570225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b39734a1-7390-4169-8095-51d614751e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:20.576922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6fb5f-b448-4e68-840f-4fee70e6b8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:20.578063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6fb5f-b448-4e68-840f-4fee70e6b8f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:20.578063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6fb5f-b448-4e68-840f-4fee70e6b8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:20.578270 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6fb5f-b448-4e68-840f-4fee70e6b8f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:20.593722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6fb5f-b448-4e68-840f-4fee70e6b8f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:20.613483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6fb5f-b448-4e68-840f-4fee70e6b8f4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:20.692040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 879/1748] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:20 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:20.695753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6fb5f-b448-4e68-840f-4fee70e6b8f4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:20.706898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6ee6fb5f-b448-4e68-840f-4fee70e6b8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:20.737819 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6fb5f-b448-4e68-840f-4fee70e6b8f4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:20.754548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:20.755208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:20.755480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:20.756357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] PATH_INFO: `/v3/projects/e81bf8c903a44061be26a1059e9665cb` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:20.757319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81bf8c903a44061be26a1059e9665cb)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:20.759458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:20.781419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:20.884232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 870/1749] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:20.885260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39734a1-7390-4169-8095-51d614751e75 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:20.888299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81bf8c903a44061be26a1059e9665cb', 'request_id': 'req-812a9694-4c27-473f-96d3-7f19eb371e1d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:20.895739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b39734a1-7390-4169-8095-51d614751e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:20.895956 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81bf8c903a44061be26a1059e9665cb', 'request_id': 'req-812a9694-4c27-473f-96d3-7f19eb371e1d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:20.905042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81bf8c903a44061be26a1059e9665cb', 'request_id': 'req-812a9694-4c27-473f-96d3-7f19eb371e1d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:20.940270 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] The token cache is being invalidate because project e81bf8c903a44061be26a1059e9665cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:20.940568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:20.943258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:20.950673 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 880/1750] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:20 2022] DELETE /identity/v3/projects/e81bf8c903a44061be26a1059e9665cb => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:20.951663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:20.953067 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:20.967664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39734a1-7390-4169-8095-51d614751e75 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:21.037477 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:21.106766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a9694-4c27-473f-96d3-7f19eb371e1d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:21.111022 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb6095-3ff1-4348-b98e-735efb628b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:21.111747 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb6095-3ff1-4348-b98e-735efb628b83 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:21.112022 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb6095-3ff1-4348-b98e-735efb628b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:21.112338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb6095-3ff1-4348-b98e-735efb628b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:21.114293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fbcb6095-3ff1-4348-b98e-735efb628b83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:21.116161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea331aa-6cfa-4a2d-b072-4f0c56c8d29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:21.116794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea331aa-6cfa-4a2d-b072-4f0c56c8d29f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:21.117071 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea331aa-6cfa-4a2d-b072-4f0c56c8d29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:21.117341 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea331aa-6cfa-4a2d-b072-4f0c56c8d29f admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:21.117982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea331aa-6cfa-4a2d-b072-4f0c56c8d29f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:21.121646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea331aa-6cfa-4a2d-b072-4f0c56c8d29f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:21.150872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb6095-3ff1-4348-b98e-735efb628b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:21.152575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb6095-3ff1-4348-b98e-735efb628b83 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:21.165365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 871/1751] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:20 2022] POST /identity/v3/projects => generated 365 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:21.166392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea331aa-6cfa-4a2d-b072-4f0c56c8d29f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:21.168311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1ea331aa-6cfa-4a2d-b072-4f0c56c8d29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:21.181043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 881/1752] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:21.182677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942f7cf-af10-4f20-a9d5-3ef69abadc6b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:21.183112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942f7cf-af10-4f20-a9d5-3ef69abadc6b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:21.183536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942f7cf-af10-4f20-a9d5-3ef69abadc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:21.231238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0942f7cf-af10-4f20-a9d5-3ef69abadc6b None None] MFA Rules not processed for user `1dde550901f04ea695b85db4e41d9628`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:21.232715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0942f7cf-af10-4f20-a9d5-3ef69abadc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:21.292806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea331aa-6cfa-4a2d-b072-4f0c56c8d29f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:21.308570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c159245-df43-4446-bbed-8deeac9a5056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:21.309422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c159245-df43-4446-bbed-8deeac9a5056 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:21.309753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c159245-df43-4446-bbed-8deeac9a5056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:21.310029 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c159245-df43-4446-bbed-8deeac9a5056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:21.312203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c159245-df43-4446-bbed-8deeac9a5056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:21.318497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c159245-df43-4446-bbed-8deeac9a5056 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:21.373645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 872/1753] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:21.376193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c159245-df43-4446-bbed-8deeac9a5056 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:21.379232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3c159245-df43-4446-bbed-8deeac9a5056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:21.410209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 882/1754] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:21.432009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942f7cf-af10-4f20-a9d5-3ef69abadc6b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:21.436942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0942f7cf-af10-4f20-a9d5-3ef69abadc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:21.450945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c159245-df43-4446-bbed-8deeac9a5056 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:21.496429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:21.499214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:21.499491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:21.504047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] PATH_INFO: `/v3/users/70bbfc114c314b8d837c16a8cb9b9712` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:21.510387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70bbfc114c314b8d837c16a8cb9b9712)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:21.521474 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:21.541164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942f7cf-af10-4f20-a9d5-3ef69abadc6b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:21.567550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfee9c1-71b1-47ac-8d86-e0c350e8bb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:21.569644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfee9c1-71b1-47ac-8d86-e0c350e8bb7f admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:21.569931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfee9c1-71b1-47ac-8d86-e0c350e8bb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:21.571579 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfee9c1-71b1-47ac-8d86-e0c350e8bb7f admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:21.572257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfee9c1-71b1-47ac-8d86-e0c350e8bb7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:21.577711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfee9c1-71b1-47ac-8d86-e0c350e8bb7f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:21.641606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 883/1755] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:21 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:21.645689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfee9c1-71b1-47ac-8d86-e0c350e8bb7f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:21.675029 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70bbfc114c314b8d837c16a8cb9b9712'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:21.679594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70bbfc114c314b8d837c16a8cb9b9712'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:21.687136 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfee9c1-71b1-47ac-8d86-e0c350e8bb7f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:21.700878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70bbfc114c314b8d837c16a8cb9b9712'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:21.704889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6355c-58ce-4dc3-9181-cdb8f7191d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:21.705934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6355c-58ce-4dc3-9181-cdb8f7191d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:21.706415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6355c-58ce-4dc3-9181-cdb8f7191d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:21.707961 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6355c-58ce-4dc3-9181-cdb8f7191d54 admin admin] PATH_INFO: `/v3/projects/16ae1fe7dc6449ac9dd662313d7a3d76/users/00e3e3996ccd432a80e7334b724da992/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:21.718710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70bbfc114c314b8d837c16a8cb9b9712'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:21.745819 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70bbfc114c314b8d837c16a8cb9b9712'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:21.761557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6355c-58ce-4dc3-9181-cdb8f7191d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16ae1fe7dc6449ac9dd662313d7a3d76, user_id=00e3e3996ccd432a80e7334b724da992, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:21.772016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 873/1756] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:21 2022] DELETE /identity/v3/users/70bbfc114c314b8d837c16a8cb9b9712 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:21.774270 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6355c-58ce-4dc3-9181-cdb8f7191d54 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:21.776032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:21.779701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:21.826039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 884/1757] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:21 2022] PUT /identity/v3/projects/16ae1fe7dc6449ac9dd662313d7a3d76/users/00e3e3996ccd432a80e7334b724da992/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:21.826833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6355c-58ce-4dc3-9181-cdb8f7191d54 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:21.837415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-bdf6355c-58ce-4dc3-9181-cdb8f7191d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:21.870508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:21.909419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:22.040188 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347475af-0d1b-4e7f-bb30-bdfaae44f4f9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:22.073932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6355c-58ce-4dc3-9181-cdb8f7191d54 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:22.083875 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70591e4a-35a3-4f8a-8cb5-910b29ae5b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:22.084650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591e4a-35a3-4f8a-8cb5-910b29ae5b29 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:22.085173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591e4a-35a3-4f8a-8cb5-910b29ae5b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:22.085641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591e4a-35a3-4f8a-8cb5-910b29ae5b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:22.087218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70591e4a-35a3-4f8a-8cb5-910b29ae5b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:22.092451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70591e4a-35a3-4f8a-8cb5-910b29ae5b29 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:22.116905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 885/1758] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:22.116905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70591e4a-35a3-4f8a-8cb5-910b29ae5b29 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:22.117640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-70591e4a-35a3-4f8a-8cb5-910b29ae5b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:22.133935 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70591e4a-35a3-4f8a-8cb5-910b29ae5b29 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:22.140872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438c003-7386-420c-96eb-5a05c40f1922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:22.141645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438c003-7386-420c-96eb-5a05c40f1922 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:22.143171 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438c003-7386-420c-96eb-5a05c40f1922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:22.143499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438c003-7386-420c-96eb-5a05c40f1922 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:22.144353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438c003-7386-420c-96eb-5a05c40f1922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:22.146190 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9438c003-7386-420c-96eb-5a05c40f1922 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:22.166805 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 886/1759] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:22.168132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438c003-7386-420c-96eb-5a05c40f1922 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:22.193163 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438c003-7386-420c-96eb-5a05c40f1922 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:22.197997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:22.198915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:22.199284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:22.199617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] PATH_INFO: `/v3/projects/aec32d84a6fa47a3bc078bb3667a2884` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:22.201247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aec32d84a6fa47a3bc078bb3667a2884)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:22.204027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:22.205477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c383e0-e1f6-450d-8068-4d4c7d38788e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:22.207524 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c383e0-e1f6-450d-8068-4d4c7d38788e admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:22.209360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c383e0-e1f6-450d-8068-4d4c7d38788e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:22.210449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c383e0-e1f6-450d-8068-4d4c7d38788e admin admin] PATH_INFO: `/v3/projects/15da4805ff5d4f259713a8d83caa1a0f/users/547eeae85f524d8abbf5ac7893fd89a2/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:22.247834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c383e0-e1f6-450d-8068-4d4c7d38788e admin admin] RBAC: Authorizing `identity:create_grant(project_id=15da4805ff5d4f259713a8d83caa1a0f, user_id=547eeae85f524d8abbf5ac7893fd89a2, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:22.251080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c383e0-e1f6-450d-8068-4d4c7d38788e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:22.279070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 887/1760] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:22 2022] PUT /identity/v3/projects/15da4805ff5d4f259713a8d83caa1a0f/users/547eeae85f524d8abbf5ac7893fd89a2/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:22.280285 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c383e0-e1f6-450d-8068-4d4c7d38788e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:22.287180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aec32d84a6fa47a3bc078bb3667a2884', 'request_id': 'req-4db7fc00-bb50-4504-9b30-ffb98f618567'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:22.296124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aec32d84a6fa47a3bc078bb3667a2884', 'request_id': 'req-4db7fc00-bb50-4504-9b30-ffb98f618567'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:22.307699 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aec32d84a6fa47a3bc078bb3667a2884', 'request_id': 'req-4db7fc00-bb50-4504-9b30-ffb98f618567'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:22.335781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] The token cache is being invalidate because project aec32d84a6fa47a3bc078bb3667a2884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:22.338570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:22.341151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:22.347456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 874/1761] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:21 2022] DELETE /identity/v3/projects/aec32d84a6fa47a3bc078bb3667a2884 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:22.348342 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:22.353932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:22.465026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c383e0-e1f6-450d-8068-4d4c7d38788e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:22.467330 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7fc00-bb50-4504-9b30-ffb98f618567 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:22.471189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f3b23-47c7-48e3-a212-915cdbaa9477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:22.472671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f3b23-47c7-48e3-a212-915cdbaa9477 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:22.473806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f3b23-47c7-48e3-a212-915cdbaa9477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:22.474271 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f3b23-47c7-48e3-a212-915cdbaa9477 admin admin] PATH_INFO: `/v3/projects/16ae1fe7dc6449ac9dd662313d7a3d76/users/00e3e3996ccd432a80e7334b724da992/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:22.474803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2483aa8-2a38-4473-8ab9-e15680bb8689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:22.475756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2483aa8-2a38-4473-8ab9-e15680bb8689 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:22.476039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2483aa8-2a38-4473-8ab9-e15680bb8689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:22.476306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2483aa8-2a38-4473-8ab9-e15680bb8689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:22.476949 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2483aa8-2a38-4473-8ab9-e15680bb8689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:22.477417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f3b23-47c7-48e3-a212-915cdbaa9477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16ae1fe7dc6449ac9dd662313d7a3d76, user_id=00e3e3996ccd432a80e7334b724da992, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:22.478862 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2483aa8-2a38-4473-8ab9-e15680bb8689 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:22.482497 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f3b23-47c7-48e3-a212-915cdbaa9477 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:22.487235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:22.501690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 875/1762] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:22.501690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2483aa8-2a38-4473-8ab9-e15680bb8689 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:22.503790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a2483aa8-2a38-4473-8ab9-e15680bb8689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:22.522403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 888/1763] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:22 2022] PUT /identity/v3/projects/16ae1fe7dc6449ac9dd662313d7a3d76/users/00e3e3996ccd432a80e7334b724da992/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:22.525050 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f3b23-47c7-48e3-a212-915cdbaa9477 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:22.538272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2483aa8-2a38-4473-8ab9-e15680bb8689 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:22.691401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:22.701484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f3b23-47c7-48e3-a212-915cdbaa9477 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:22.702194 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5210c-d491-450b-b5fa-69f7115c31b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:22.702470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5210c-d491-450b-b5fa-69f7115c31b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:22.702749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5210c-d491-450b-b5fa-69f7115c31b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:22.703016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5210c-d491-450b-b5fa-69f7115c31b6 admin admin] PATH_INFO: `/v3/users/d950b6f57850439881bab3e0a5e99b99` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:22.707911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5210c-d491-450b-b5fa-69f7115c31b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d950b6f57850439881bab3e0a5e99b99)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:22.709560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f502fa-1ed9-4b98-838f-ed57be667ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:22.709851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5210c-d491-450b-b5fa-69f7115c31b6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:22.710609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f502fa-1ed9-4b98-838f-ed57be667ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:22.711070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f502fa-1ed9-4b98-838f-ed57be667ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:22.711616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f502fa-1ed9-4b98-838f-ed57be667ee3 admin admin] PATH_INFO: `/v3/projects/15da4805ff5d4f259713a8d83caa1a0f/users/547eeae85f524d8abbf5ac7893fd89a2/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:22.713890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f502fa-1ed9-4b98-838f-ed57be667ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15da4805ff5d4f259713a8d83caa1a0f, user_id=547eeae85f524d8abbf5ac7893fd89a2, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:22.718713 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f502fa-1ed9-4b98-838f-ed57be667ee3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:22.746987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 889/1764] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:22 2022] PUT /identity/v3/projects/15da4805ff5d4f259713a8d83caa1a0f/users/547eeae85f524d8abbf5ac7893fd89a2/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:22.748749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45afc42-93cd-4a29-81b4-ccfc2a207fc3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:22.748996 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45afc42-93cd-4a29-81b4-ccfc2a207fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:22.749273 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45afc42-93cd-4a29-81b4-ccfc2a207fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:22.862546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c9f5210c-d491-450b-b5fa-69f7115c31b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd950b6f57850439881bab3e0a5e99b99'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:22.863391 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c9f5210c-d491-450b-b5fa-69f7115c31b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd950b6f57850439881bab3e0a5e99b99'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:22.865466 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e45afc42-93cd-4a29-81b4-ccfc2a207fc3 None None] MFA Rules not processed for user `00e3e3996ccd432a80e7334b724da992`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:22.867526 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e45afc42-93cd-4a29-81b4-ccfc2a207fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:22.889621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c9f5210c-d491-450b-b5fa-69f7115c31b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd950b6f57850439881bab3e0a5e99b99'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:22.907346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c9f5210c-d491-450b-b5fa-69f7115c31b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd950b6f57850439881bab3e0a5e99b99'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:22.931297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c9f5210c-d491-450b-b5fa-69f7115c31b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd950b6f57850439881bab3e0a5e99b99'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:22.940330 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 876/1765] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:22 2022] DELETE /identity/v3/users/d950b6f57850439881bab3e0a5e99b99 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:22.942337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1b65b-9469-4e91-a8f0-5c2522c10e62 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:22.942618 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1b65b-9469-4e91-a8f0-5c2522c10e62 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:22.942884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1b65b-9469-4e91-a8f0-5c2522c10e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:23.062692 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a0a1b65b-9469-4e91-a8f0-5c2522c10e62 None None] MFA Rules not processed for user `547eeae85f524d8abbf5ac7893fd89a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:23.069128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a0a1b65b-9469-4e91-a8f0-5c2522c10e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:23.151784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 890/1766] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:23.152748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45afc42-93cd-4a29-81b4-ccfc2a207fc3 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:23.159051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e45afc42-93cd-4a29-81b4-ccfc2a207fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:23.331943 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 877/1767] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:23.334180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1b65b-9469-4e91-a8f0-5c2522c10e62 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:23.346957 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a0a1b65b-9469-4e91-a8f0-5c2522c10e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:23.361139 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:23.380969 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45afc42-93cd-4a29-81b4-ccfc2a207fc3 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:23.563499 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1b65b-9469-4e91-a8f0-5c2522c10e62 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:23.589906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:23.592329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:23.595991 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:23.596435 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] PATH_INFO: `/v3/projects/3a7cbd60b036481ea69455bd5bea1a27` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:23.599625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:23.601602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:23.602643 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a7cbd60b036481ea69455bd5bea1a27)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:23.603183 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:23.603680 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] PATH_INFO: `/v3/projects/0b7fcad3ea934d18b26586a80661b549` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:23.606171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:23.607060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b7fcad3ea934d18b26586a80661b549)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:23.611435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:23.713047 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:23.813615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b7fcad3ea934d18b26586a80661b549', 'request_id': 'req-637ae1ca-9cec-4795-9389-b6ffc9b36aba'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:23.823918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b7fcad3ea934d18b26586a80661b549', 'request_id': 'req-637ae1ca-9cec-4795-9389-b6ffc9b36aba'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:23.850872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a7cbd60b036481ea69455bd5bea1a27', 'request_id': 'req-5022c4b7-06e6-4668-8636-e398dbfc32bc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:23.867081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a7cbd60b036481ea69455bd5bea1a27', 'request_id': 'req-5022c4b7-06e6-4668-8636-e398dbfc32bc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:23.868340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b7fcad3ea934d18b26586a80661b549', 'request_id': 'req-637ae1ca-9cec-4795-9389-b6ffc9b36aba'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:23.878440 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a7cbd60b036481ea69455bd5bea1a27', 'request_id': 'req-5022c4b7-06e6-4668-8636-e398dbfc32bc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:23.920370 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] The token cache is being invalidate because project 3a7cbd60b036481ea69455bd5bea1a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:23.920805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:23.925989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:23.928409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 878/1768] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:23 2022] DELETE /identity/v3/projects/3a7cbd60b036481ea69455bd5bea1a27 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:23.929224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:23.932392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] The token cache is being invalidate because project 0b7fcad3ea934d18b26586a80661b549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:23.935116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:23.936024 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:23.945215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:23.947598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 891/1769] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:23 2022] DELETE /identity/v3/projects/0b7fcad3ea934d18b26586a80661b549 => generated 0 bytes in 795 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:23.948731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:23.955144 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:24.103493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ae1ca-9cec-4795-9389-b6ffc9b36aba admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:24.108079 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022c4b7-06e6-4668-8636-e398dbfc32bc admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:24.115056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd48de7-eb81-4d86-8f1b-666446907f2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:24.115878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd48de7-eb81-4d86-8f1b-666446907f2e service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:24.116264 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd48de7-eb81-4d86-8f1b-666446907f2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:24.116392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc76a5-909e-4376-bda7-9318e14bdd84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:24.116728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd48de7-eb81-4d86-8f1b-666446907f2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:24.116892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc76a5-909e-4376-bda7-9318e14bdd84 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:24.117353 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc76a5-909e-4376-bda7-9318e14bdd84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:24.117633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc76a5-909e-4376-bda7-9318e14bdd84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:24.123670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d4fc76a5-909e-4376-bda7-9318e14bdd84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:24.125036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9cd48de7-eb81-4d86-8f1b-666446907f2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:24.235913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:24.266115 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc76a5-909e-4376-bda7-9318e14bdd84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:24.267772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc76a5-909e-4376-bda7-9318e14bdd84 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:24.307928 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 879/1770] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:24.309544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd48de7-eb81-4d86-8f1b-666446907f2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:24.310880 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc76a5-909e-4376-bda7-9318e14bdd84 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:24.311321 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd48de7-eb81-4d86-8f1b-666446907f2e service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:24.316862 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d4fc76a5-909e-4376-bda7-9318e14bdd84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:24.333197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 892/1771] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:23 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:24.335077 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea30d0-cb6e-4deb-8f98-ed98a7f68a84 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:24.335438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea30d0-cb6e-4deb-8f98-ed98a7f68a84 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:24.335762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea30d0-cb6e-4deb-8f98-ed98a7f68a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:24.378734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d8ea30d0-cb6e-4deb-8f98-ed98a7f68a84 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:24.380619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d8ea30d0-cb6e-4deb-8f98-ed98a7f68a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:24.488507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 893/1772] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:24.489523 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea30d0-cb6e-4deb-8f98-ed98a7f68a84 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:24.492158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d8ea30d0-cb6e-4deb-8f98-ed98a7f68a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:24.503642 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:24.509967 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea30d0-cb6e-4deb-8f98-ed98a7f68a84 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:24.525293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f415e7e-f616-491b-80e5-944b303e31d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:24.526336 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f415e7e-f616-491b-80e5-944b303e31d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:24.526778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f415e7e-f616-491b-80e5-944b303e31d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:24.527105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f415e7e-f616-491b-80e5-944b303e31d6 admin admin] PATH_INFO: `/v3/projects/f28db939503d4fe2b7969a7dee575ed3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:24.529074 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f415e7e-f616-491b-80e5-944b303e31d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f28db939503d4fe2b7969a7dee575ed3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:24.531503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f415e7e-f616-491b-80e5-944b303e31d6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:24.566526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc76a5-909e-4376-bda7-9318e14bdd84 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:24.598279 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:24.598937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:24.599214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:24.599485 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] PATH_INFO: `/v3/users/2ee54bde58874323accf29efa724e0d1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:24.606482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ee54bde58874323accf29efa724e0d1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:24.616893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:24.674164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2f415e7e-f616-491b-80e5-944b303e31d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f28db939503d4fe2b7969a7dee575ed3', 'request_id': 'req-2f415e7e-f616-491b-80e5-944b303e31d6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:24.699688 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2f415e7e-f616-491b-80e5-944b303e31d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f28db939503d4fe2b7969a7dee575ed3', 'request_id': 'req-2f415e7e-f616-491b-80e5-944b303e31d6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:24.720228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2f415e7e-f616-491b-80e5-944b303e31d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f28db939503d4fe2b7969a7dee575ed3', 'request_id': 'req-2f415e7e-f616-491b-80e5-944b303e31d6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:24.752464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2f415e7e-f616-491b-80e5-944b303e31d6 admin admin] The token cache is being invalidate because project f28db939503d4fe2b7969a7dee575ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:24.753263 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2f415e7e-f616-491b-80e5-944b303e31d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:24.754261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2f415e7e-f616-491b-80e5-944b303e31d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:24.755420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ee54bde58874323accf29efa724e0d1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:24.756961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ee54bde58874323accf29efa724e0d1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:24.764912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 894/1773] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:24 2022] DELETE /identity/v3/projects/f28db939503d4fe2b7969a7dee575ed3 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:24.770245 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa0f82-cc7a-462c-a51a-c08772aac01a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:24.770588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa0f82-cc7a-462c-a51a-c08772aac01a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:24.770901 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa0f82-cc7a-462c-a51a-c08772aac01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:24.781546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ee54bde58874323accf29efa724e0d1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:24.807460 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ee54bde58874323accf29efa724e0d1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:24.832814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b1aa0f82-cc7a-462c-a51a-c08772aac01a None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:24.834458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b1aa0f82-cc7a-462c-a51a-c08772aac01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:24.847081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ee54bde58874323accf29efa724e0d1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:24.875500 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 880/1774] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:24 2022] DELETE /identity/v3/users/2ee54bde58874323accf29efa724e0d1 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:24.878960 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:24.881939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:25.025603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 895/1775] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:25.031609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c195791-e0d1-4486-91ec-d4e190f9afa4 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:25.031920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c195791-e0d1-4486-91ec-d4e190f9afa4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:25.032193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c195791-e0d1-4486-91ec-d4e190f9afa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:25.064864 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ef701-8715-48b8-9a5b-c3b8899018c5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:25.076253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5377b1ca-939d-449b-8d7a-dc3548b20f8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:25.077272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377b1ca-939d-449b-8d7a-dc3548b20f8e service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:25.077546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377b1ca-939d-449b-8d7a-dc3548b20f8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:25.077816 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5377b1ca-939d-449b-8d7a-dc3548b20f8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:25.086872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5377b1ca-939d-449b-8d7a-dc3548b20f8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:25.175208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:25.185737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2c195791-e0d1-4486-91ec-d4e190f9afa4 None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:25.187198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2c195791-e0d1-4486-91ec-d4e190f9afa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:25.327680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377b1ca-939d-449b-8d7a-dc3548b20f8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:25.328474 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5377b1ca-939d-449b-8d7a-dc3548b20f8e service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:25.353639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 881/1776] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:25.374153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dd193-1168-4270-8be5-633eaf9eef4e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:25.374916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:25.375946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dd193-1168-4270-8be5-633eaf9eef4e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:25.376255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dd193-1168-4270-8be5-633eaf9eef4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:25.383165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 896/1777] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:24:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:25.384189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c195791-e0d1-4486-91ec-d4e190f9afa4 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:25.484125 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fe9dd193-1168-4270-8be5-633eaf9eef4e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:25.491153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fe9dd193-1168-4270-8be5-633eaf9eef4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:25.714439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c195791-e0d1-4486-91ec-d4e190f9afa4 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:25.724625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf1fdf-be25-419e-b10c-5f309640bc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:25.725499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf1fdf-be25-419e-b10c-5f309640bc23 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:25.725954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf1fdf-be25-419e-b10c-5f309640bc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:25.727017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf1fdf-be25-419e-b10c-5f309640bc23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:25.727910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf1fdf-be25-419e-b10c-5f309640bc23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:25.728410 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf1fdf-be25-419e-b10c-5f309640bc23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:25.730632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf1fdf-be25-419e-b10c-5f309640bc23 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:25.770863 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 882/1778] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:25.770863 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dd193-1168-4270-8be5-633eaf9eef4e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:25.770863 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fe9dd193-1168-4270-8be5-633eaf9eef4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:25.792055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 897/1779] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:25.796903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ddc6c-77a0-4a30-b8d9-b403e644a9a3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:25.797225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ddc6c-77a0-4a30-b8d9-b403e644a9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:25.798437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ddc6c-77a0-4a30-b8d9-b403e644a9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:25.858335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c71ddc6c-77a0-4a30-b8d9-b403e644a9a3 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:25.859910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c71ddc6c-77a0-4a30-b8d9-b403e644a9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:25.872363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 898/1780] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:25.873248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ddc6c-77a0-4a30-b8d9-b403e644a9a3 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:25.900544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ddc6c-77a0-4a30-b8d9-b403e644a9a3 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:25.906943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7ca11-fcd4-4773-8789-07942df46a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:25.907659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7ca11-fcd4-4773-8789-07942df46a9f admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:25.907949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7ca11-fcd4-4773-8789-07942df46a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:25.908195 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7ca11-fcd4-4773-8789-07942df46a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:25.908828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7ca11-fcd4-4773-8789-07942df46a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:25.910859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d7ca11-fcd4-4773-8789-07942df46a9f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:25.959046 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 899/1781] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:25 2022] POST /identity/v3/projects => generated 367 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:25.963813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7ca11-fcd4-4773-8789-07942df46a9f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:26.115226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7ca11-fcd4-4773-8789-07942df46a9f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:26.134200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233f144-7266-419f-9f60-18f80c64df03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:26.135710 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233f144-7266-419f-9f60-18f80c64df03 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:26.135836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233f144-7266-419f-9f60-18f80c64df03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:26.136102 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233f144-7266-419f-9f60-18f80c64df03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:26.139699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233f144-7266-419f-9f60-18f80c64df03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:26.140017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233f144-7266-419f-9f60-18f80c64df03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:26.147484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233f144-7266-419f-9f60-18f80c64df03 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:26.151441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dd193-1168-4270-8be5-633eaf9eef4e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:26.192467 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 900/1782] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:26.197270 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233f144-7266-419f-9f60-18f80c64df03 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:26.222298 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233f144-7266-419f-9f60-18f80c64df03 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:26.235756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22614487-1ed2-45d5-afda-8fe1f5bc0420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:26.238891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22614487-1ed2-45d5-afda-8fe1f5bc0420 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:26.239176 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22614487-1ed2-45d5-afda-8fe1f5bc0420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:26.239444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22614487-1ed2-45d5-afda-8fe1f5bc0420 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:26.240092 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22614487-1ed2-45d5-afda-8fe1f5bc0420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:26.244856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22614487-1ed2-45d5-afda-8fe1f5bc0420 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:26.295468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:26.323134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 901/1783] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:26 2022] POST /identity/v3/users => generated 284 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:26.329013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be623911-9956-402f-b677-1477378f835f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:26.329300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be623911-9956-402f-b677-1477378f835f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:26.329572 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be623911-9956-402f-b677-1477378f835f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:26.350124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ade96b-72cc-4f37-84e7-78c3c8c94c5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:26.350831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ade96b-72cc-4f37-84e7-78c3c8c94c5d service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:26.351136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ade96b-72cc-4f37-84e7-78c3c8c94c5d service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:26.351452 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ade96b-72cc-4f37-84e7-78c3c8c94c5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:26.397321 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-be623911-9956-402f-b677-1477378f835f None None] MFA Rules not processed for user `5d906452199844149966c9c86db4684e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:26.399185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-be623911-9956-402f-b677-1477378f835f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:26.494280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:26.517780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ade96b-72cc-4f37-84e7-78c3c8c94c5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:26.524794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ade96b-72cc-4f37-84e7-78c3c8c94c5d service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:26.633078 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 902/1785] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Sep 27 17:24:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:26.633284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 883/1785] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:24:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 868 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:24:26.634297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ade96b-72cc-4f37-84e7-78c3c8c94c5d service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:26.635789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be623911-9956-402f-b677-1477378f835f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:26.670718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ade96b-72cc-4f37-84e7-78c3c8c94c5d service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:26.676664 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be623911-9956-402f-b677-1477378f835f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:26.691807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff24d7f-157b-4bef-a7cc-0b97bd8309db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:26.692461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff24d7f-157b-4bef-a7cc-0b97bd8309db admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:26.692785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff24d7f-157b-4bef-a7cc-0b97bd8309db admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:26.692986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff24d7f-157b-4bef-a7cc-0b97bd8309db admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:26.693666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff24d7f-157b-4bef-a7cc-0b97bd8309db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:26.698353 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff24d7f-157b-4bef-a7cc-0b97bd8309db admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:26.708367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84397fb-71a9-4142-9787-2a971f09f276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:26.709043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84397fb-71a9-4142-9787-2a971f09f276 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:26.709486 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84397fb-71a9-4142-9787-2a971f09f276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:26.709748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84397fb-71a9-4142-9787-2a971f09f276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:26.710510 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84397fb-71a9-4142-9787-2a971f09f276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:26.712173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84397fb-71a9-4142-9787-2a971f09f276 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:26.761418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 903/1786] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:26.762908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84397fb-71a9-4142-9787-2a971f09f276 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:26.781944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 884/1787] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:26 2022] POST /identity/v3/projects => generated 367 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:26.783078 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff24d7f-157b-4bef-a7cc-0b97bd8309db admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:26.785780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b84397fb-71a9-4142-9787-2a971f09f276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:26.826300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84397fb-71a9-4142-9787-2a971f09f276 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:26.990999 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767cdf3f-03ae-4ba5-ad0d-038b84b7547e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:26.992019 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cdf3f-03ae-4ba5-ad0d-038b84b7547e service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:26.993922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cdf3f-03ae-4ba5-ad0d-038b84b7547e service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:26.994460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cdf3f-03ae-4ba5-ad0d-038b84b7547e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:27.024292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767cdf3f-03ae-4ba5-ad0d-038b84b7547e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:27.024692 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff24d7f-157b-4bef-a7cc-0b97bd8309db admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:27.026050 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767cdf3f-03ae-4ba5-ad0d-038b84b7547e service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:27.047730 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b9cd5-9413-4087-8c84-fb52ce83810f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:27.048392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b9cd5-9413-4087-8c84-fb52ce83810f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:27.048770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b9cd5-9413-4087-8c84-fb52ce83810f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:27.049055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b9cd5-9413-4087-8c84-fb52ce83810f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:27.049750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b9cd5-9413-4087-8c84-fb52ce83810f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:27.051493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b9cd5-9413-4087-8c84-fb52ce83810f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:27.069299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 904/1788] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:27.070489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767cdf3f-03ae-4ba5-ad0d-038b84b7547e service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:27.104046 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 885/1789] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:27.105344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b9cd5-9413-4087-8c84-fb52ce83810f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:27.112360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767cdf3f-03ae-4ba5-ad0d-038b84b7547e service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:27.133448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a95836-d1c0-46a6-8ece-24e96115b8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:27.134563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a95836-d1c0-46a6-8ece-24e96115b8e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:27.134841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a95836-d1c0-46a6-8ece-24e96115b8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:27.137400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a95836-d1c0-46a6-8ece-24e96115b8e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:27.140415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a95836-d1c0-46a6-8ece-24e96115b8e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:27.146447 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a95836-d1c0-46a6-8ece-24e96115b8e6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:27.162866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b9cd5-9413-4087-8c84-fb52ce83810f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:27.179253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e75029-0993-42e4-b9bf-e9701c235042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:27.179979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e75029-0993-42e4-b9bf-e9701c235042 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:27.180338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e75029-0993-42e4-b9bf-e9701c235042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:27.180639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e75029-0993-42e4-b9bf-e9701c235042 admin admin] PATH_INFO: `/v3/projects/fd4a3ae5c3ee44c9829322a938a836e4/users/2c5f043c4e464ce4ac6e570ccbbadc34/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:27.218778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 905/1790] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:27 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:27.222456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a95836-d1c0-46a6-8ece-24e96115b8e6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:27.226021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-48a95836-d1c0-46a6-8ece-24e96115b8e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:27.239461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e75029-0993-42e4-b9bf-e9701c235042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd4a3ae5c3ee44c9829322a938a836e4, user_id=2c5f043c4e464ce4ac6e570ccbbadc34, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:27.242355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e75029-0993-42e4-b9bf-e9701c235042 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:27.253501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a95836-d1c0-46a6-8ece-24e96115b8e6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:27.266670 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:27.267158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:27.267440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:27.267713 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8 admin admin] PATH_INFO: `/v3/projects/69840972df1f48e58f12c14c021d5009` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:27.271455 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69840972df1f48e58f12c14c021d5009)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:27.276825 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 886/1791] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:27 2022] PUT /identity/v3/projects/fd4a3ae5c3ee44c9829322a938a836e4/users/2c5f043c4e464ce4ac6e570ccbbadc34/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:27.276961 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:27.291547 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e75029-0993-42e4-b9bf-e9701c235042 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:27.416484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:27.433915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69840972df1f48e58f12c14c021d5009', 'request_id': 'req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:27.455278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69840972df1f48e58f12c14c021d5009', 'request_id': 'req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:27.476554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69840972df1f48e58f12c14c021d5009', 'request_id': 'req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:27.515751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e75029-0993-42e4-b9bf-e9701c235042 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:27.516119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8 admin admin] The token cache is being invalidate because project 69840972df1f48e58f12c14c021d5009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:27.516464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:27.519493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1a01cb3-6480-4f04-a57a-c47d5d1b04c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:27.524423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 906/1792] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:27 2022] DELETE /identity/v3/projects/69840972df1f48e58f12c14c021d5009 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:27.525730 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44886-7deb-46be-b2f9-b4939459f78d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:27.526010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44886-7deb-46be-b2f9-b4939459f78d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:27.526304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b44886-7deb-46be-b2f9-b4939459f78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:27.636122 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-79b44886-7deb-46be-b2f9-b4939459f78d None None] MFA Rules not processed for user `2c5f043c4e464ce4ac6e570ccbbadc34`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:27.641854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-79b44886-7deb-46be-b2f9-b4939459f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:27.778894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5c166-aa0e-4c7f-9ca4-af44dfe981c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:27.779522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5c166-aa0e-4c7f-9ca4-af44dfe981c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:27.779788 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5c166-aa0e-4c7f-9ca4-af44dfe981c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:27.780055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5c166-aa0e-4c7f-9ca4-af44dfe981c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:27.780708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5c166-aa0e-4c7f-9ca4-af44dfe981c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:27.785339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5c166-aa0e-4c7f-9ca4-af44dfe981c6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:27.819228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:27.821509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 887/1793] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 545 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:27.823117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86844-0b29-47e8-8635-62dc2405f00e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:27.823403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86844-0b29-47e8-8635-62dc2405f00e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:27.823671 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86844-0b29-47e8-8635-62dc2405f00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:27.875369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-faa86844-0b29-47e8-8635-62dc2405f00e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:27.877122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-faa86844-0b29-47e8-8635-62dc2405f00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:27.902466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 888/1794] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:27.902466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86844-0b29-47e8-8635-62dc2405f00e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:27.902466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-faa86844-0b29-47e8-8635-62dc2405f00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:27.910998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 907/1795] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:27.912099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5889f-ce5f-4f80-9015-ce2732f34719 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:27.913072 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5889f-ce5f-4f80-9015-ce2732f34719 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:27.915653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5889f-ce5f-4f80-9015-ce2732f34719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:27.919769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86844-0b29-47e8-8635-62dc2405f00e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:27.926216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35b515-c8bf-440d-8fc3-9c537b7d9d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:27.927087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35b515-c8bf-440d-8fc3-9c537b7d9d5b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:27.927577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35b515-c8bf-440d-8fc3-9c537b7d9d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:27.927953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec35b515-c8bf-440d-8fc3-9c537b7d9d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:27.929014 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35b515-c8bf-440d-8fc3-9c537b7d9d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:27.931107 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec35b515-c8bf-440d-8fc3-9c537b7d9d5b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:27.954395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 889/1796] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:27.954395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35b515-c8bf-440d-8fc3-9c537b7d9d5b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:27.954395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ec35b515-c8bf-440d-8fc3-9c537b7d9d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:27.974863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f8c5889f-ce5f-4f80-9015-ce2732f34719 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:27.977592 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f8c5889f-ce5f-4f80-9015-ce2732f34719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:28.006084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 908/1797] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:28.009785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5889f-ce5f-4f80-9015-ce2732f34719 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:28.035974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5889f-ce5f-4f80-9015-ce2732f34719 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:28.049108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec35b515-c8bf-440d-8fc3-9c537b7d9d5b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:28.050415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465baf1a-e12f-4b5d-bf9e-ca9ecbe2f058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:28.051923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465baf1a-e12f-4b5d-bf9e-ca9ecbe2f058 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:28.053655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465baf1a-e12f-4b5d-bf9e-ca9ecbe2f058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:28.053655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465baf1a-e12f-4b5d-bf9e-ca9ecbe2f058 admin admin] PATH_INFO: `/v3/projects/d38fddb6eaab4f37986f373ec3b6af44/users/2b212f3dd74b4def9e588be167159419/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:28.058473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03660fcb-51cc-43a5-b98c-5dbccde9c31b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:28.059530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03660fcb-51cc-43a5-b98c-5dbccde9c31b service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:28.060654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03660fcb-51cc-43a5-b98c-5dbccde9c31b service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:28.061197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03660fcb-51cc-43a5-b98c-5dbccde9c31b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:28.080455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03660fcb-51cc-43a5-b98c-5dbccde9c31b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:28.084083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03660fcb-51cc-43a5-b98c-5dbccde9c31b service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:28.090096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465baf1a-e12f-4b5d-bf9e-ca9ecbe2f058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d38fddb6eaab4f37986f373ec3b6af44, user_id=2b212f3dd74b4def9e588be167159419, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:28.095345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465baf1a-e12f-4b5d-bf9e-ca9ecbe2f058 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:28.110254 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 890/1798] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:28.111113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03660fcb-51cc-43a5-b98c-5dbccde9c31b service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:28.139989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03660fcb-51cc-43a5-b98c-5dbccde9c31b service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:28.153386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 909/1799] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:28 2022] PUT /identity/v3/projects/d38fddb6eaab4f37986f373ec3b6af44/users/2b212f3dd74b4def9e588be167159419/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:28.175870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe26be-3241-460d-8dc2-ca2d558c4805 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:28.176180 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe26be-3241-460d-8dc2-ca2d558c4805 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:28.176449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe26be-3241-460d-8dc2-ca2d558c4805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:28.250925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5ffe26be-3241-460d-8dc2-ca2d558c4805 None None] MFA Rules not processed for user `2b212f3dd74b4def9e588be167159419`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:28.252167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5ffe26be-3241-460d-8dc2-ca2d558c4805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:28.356762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36826c-5a1e-4077-a122-abe18ce1e240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:28.362875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36826c-5a1e-4077-a122-abe18ce1e240 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:28.364595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36826c-5a1e-4077-a122-abe18ce1e240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:28.364917 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36826c-5a1e-4077-a122-abe18ce1e240 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:28.365758 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36826c-5a1e-4077-a122-abe18ce1e240 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:28.366115 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36826c-5a1e-4077-a122-abe18ce1e240 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:28.375162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36826c-5a1e-4077-a122-abe18ce1e240 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:28.412742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:28.448831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 891/1800] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:28.453520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36826c-5a1e-4077-a122-abe18ce1e240 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:28.479040 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36826c-5a1e-4077-a122-abe18ce1e240 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:28.489760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f9ff0-b0f6-4102-a417-0766eeb00a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:28.490432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f9ff0-b0f6-4102-a417-0766eeb00a98 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:28.490742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f9ff0-b0f6-4102-a417-0766eeb00a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:28.491021 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f9ff0-b0f6-4102-a417-0766eeb00a98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:28.491666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f9ff0-b0f6-4102-a417-0766eeb00a98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:28.497425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f9ff0-b0f6-4102-a417-0766eeb00a98 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:28.530682 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 910/1801] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:28.539500 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe26be-3241-460d-8dc2-ca2d558c4805 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:28.567551 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe26be-3241-460d-8dc2-ca2d558c4805 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:28.568127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 892/1802] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:28 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:28.569339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f9ff0-b0f6-4102-a417-0766eeb00a98 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:28.718609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cf409-b76e-4b46-9212-bc4a6c9af946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:28.719268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cf409-b76e-4b46-9212-bc4a6c9af946 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:28.719544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cf409-b76e-4b46-9212-bc4a6c9af946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:28.719812 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346cf409-b76e-4b46-9212-bc4a6c9af946 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:28.720472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cf409-b76e-4b46-9212-bc4a6c9af946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:28.723239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346cf409-b76e-4b46-9212-bc4a6c9af946 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:28.765359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f9ff0-b0f6-4102-a417-0766eeb00a98 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:28.782489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 911/1803] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:28 2022] POST /identity/v3/projects => generated 367 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:28.787417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cf409-b76e-4b46-9212-bc4a6c9af946 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:28.787774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc69f8-c867-47de-b44e-0bdea458d706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:28.788485 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc69f8-c867-47de-b44e-0bdea458d706 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:28.788818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc69f8-c867-47de-b44e-0bdea458d706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:28.789159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc69f8-c867-47de-b44e-0bdea458d706 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:28.796253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc69f8-c867-47de-b44e-0bdea458d706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:28.814531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc69f8-c867-47de-b44e-0bdea458d706 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:28.858617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:28.884623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 893/1804] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:28 2022] POST /identity/v3/users => generated 294 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:28.888253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc69f8-c867-47de-b44e-0bdea458d706 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:28.909346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc69f8-c867-47de-b44e-0bdea458d706 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:28.951326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346cf409-b76e-4b46-9212-bc4a6c9af946 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:28.965100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155301c-5ee4-48c3-9761-91f3ed27b0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:28.965843 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155301c-5ee4-48c3-9761-91f3ed27b0e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:28.966142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155301c-5ee4-48c3-9761-91f3ed27b0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:28.966437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155301c-5ee4-48c3-9761-91f3ed27b0e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:28.967083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155301c-5ee4-48c3-9761-91f3ed27b0e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:28.969085 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155301c-5ee4-48c3-9761-91f3ed27b0e6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:29.016170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 912/1805] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:28 2022] POST /identity/v3/users => generated 290 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:29.025569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635746e5-88c9-44df-8dd6-0ae8d39f8aa9 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:29.025861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635746e5-88c9-44df-8dd6-0ae8d39f8aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:29.026168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635746e5-88c9-44df-8dd6-0ae8d39f8aa9 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:29.027571 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 913/1806] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:24:29 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:24:29.049077 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635746e5-88c9-44df-8dd6-0ae8d39f8aa9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:29.075550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635746e5-88c9-44df-8dd6-0ae8d39f8aa9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:29.085003 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4231d-2932-4c0e-9ea1-4ec4460726a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:29.086240 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4231d-2932-4c0e-9ea1-4ec4460726a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:29.086386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4231d-2932-4c0e-9ea1-4ec4460726a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:29.086568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4231d-2932-4c0e-9ea1-4ec4460726a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:29.087255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4231d-2932-4c0e-9ea1-4ec4460726a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:29.088863 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4231d-2932-4c0e-9ea1-4ec4460726a2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:29.092914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7242749-9969-4534-855f-c6c7ce418f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:29.094108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7242749-9969-4534-855f-c6c7ce418f90 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:29.094108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7242749-9969-4534-855f-c6c7ce418f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:29.094402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7242749-9969-4534-855f-c6c7ce418f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:29.095129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7242749-9969-4534-855f-c6c7ce418f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:29.096696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7242749-9969-4534-855f-c6c7ce418f90 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:29.120926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 894/1807] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:29.121337 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 914/1808] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:29.123417 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f683339-c801-4b3f-8115-7148cd68b623 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:29.123707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f683339-c801-4b3f-8115-7148cd68b623 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:29.124032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f683339-c801-4b3f-8115-7148cd68b623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:29.126277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7242749-9969-4534-855f-c6c7ce418f90 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:29.169145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7242749-9969-4534-855f-c6c7ce418f90 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:29.201805 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de171b9a-fa83-48e9-9829-4c70700b1008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:29.203523 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de171b9a-fa83-48e9-9829-4c70700b1008 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:29.204653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de171b9a-fa83-48e9-9829-4c70700b1008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:29.204942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de171b9a-fa83-48e9-9829-4c70700b1008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:29.205614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de171b9a-fa83-48e9-9829-4c70700b1008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:29.217181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7f683339-c801-4b3f-8115-7148cd68b623 None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:29.218438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7f683339-c801-4b3f-8115-7148cd68b623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:29.219487 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de171b9a-fa83-48e9-9829-4c70700b1008 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:29.239175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 915/1809] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:29.239890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de171b9a-fa83-48e9-9829-4c70700b1008 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:29.251325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de171b9a-fa83-48e9-9829-4c70700b1008 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:29.257101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe814e8c-3fee-4d10-9071-d58648f2af24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:29.257847 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe814e8c-3fee-4d10-9071-d58648f2af24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:29.258221 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe814e8c-3fee-4d10-9071-d58648f2af24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:29.258560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe814e8c-3fee-4d10-9071-d58648f2af24 admin admin] PATH_INFO: `/v3/projects/0d0eaeb82af44bc0a9012a19adc59c7c/users/f669fdee32704583bfb471651f74226a/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:29.280508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe814e8c-3fee-4d10-9071-d58648f2af24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0eaeb82af44bc0a9012a19adc59c7c, user_id=f669fdee32704583bfb471651f74226a, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:29.283797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe814e8c-3fee-4d10-9071-d58648f2af24 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:29.310052 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 916/1810] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:29 2022] PUT /identity/v3/projects/0d0eaeb82af44bc0a9012a19adc59c7c/users/f669fdee32704583bfb471651f74226a/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:29.312612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe814e8c-3fee-4d10-9071-d58648f2af24 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:29.337236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:29.362635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 895/1811] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:24:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:29.363697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f683339-c801-4b3f-8115-7148cd68b623 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:29.377744 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f683339-c801-4b3f-8115-7148cd68b623 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:29.442702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe814e8c-3fee-4d10-9071-d58648f2af24 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:29.447924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97d6dd-354e-4384-be45-28bff0da3ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:29.448609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97d6dd-354e-4384-be45-28bff0da3ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:29.448787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6da08-a754-4a27-abd8-02662bf5105f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:29.449018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97d6dd-354e-4384-be45-28bff0da3ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:29.449543 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97d6dd-354e-4384-be45-28bff0da3ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:29.449639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6da08-a754-4a27-abd8-02662bf5105f admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:29.450306 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6da08-a754-4a27-abd8-02662bf5105f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:29.450306 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6da08-a754-4a27-abd8-02662bf5105f admin admin] PATH_INFO: `/v3/projects/3267e0059119481e81c82606a7a304ed/users/5618db388e674db4b2e35c7da6dc6911/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:29.450643 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97d6dd-354e-4384-be45-28bff0da3ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:29.453018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97d6dd-354e-4384-be45-28bff0da3ae1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:29.466471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6da08-a754-4a27-abd8-02662bf5105f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3267e0059119481e81c82606a7a304ed, user_id=5618db388e674db4b2e35c7da6dc6911, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:29.470189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 896/1812] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:29.470189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97d6dd-354e-4384-be45-28bff0da3ae1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:29.470414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6da08-a754-4a27-abd8-02662bf5105f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:29.472623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-df97d6dd-354e-4384-be45-28bff0da3ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:29.485363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 917/1813] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:29 2022] PUT /identity/v3/projects/3267e0059119481e81c82606a7a304ed/users/5618db388e674db4b2e35c7da6dc6911/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:29.486806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6da08-a754-4a27-abd8-02662bf5105f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:29.552091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:29.559467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97d6dd-354e-4384-be45-28bff0da3ae1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:29.568471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6da08-a754-4a27-abd8-02662bf5105f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:29.575942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da76e756-c357-4cc1-820b-44afe58a9d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:29.576626 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76e756-c357-4cc1-820b-44afe58a9d67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:29.576900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76e756-c357-4cc1-820b-44afe58a9d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:29.577168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da76e756-c357-4cc1-820b-44afe58a9d67 admin admin] PATH_INFO: `/v3/projects/0d0eaeb82af44bc0a9012a19adc59c7c/users/f669fdee32704583bfb471651f74226a/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:29.579324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76e756-c357-4cc1-820b-44afe58a9d67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d0eaeb82af44bc0a9012a19adc59c7c, user_id=f669fdee32704583bfb471651f74226a, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:29.582603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da76e756-c357-4cc1-820b-44afe58a9d67 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:29.597699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 918/1814] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:29 2022] PUT /identity/v3/projects/0d0eaeb82af44bc0a9012a19adc59c7c/users/f669fdee32704583bfb471651f74226a/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:29.599992 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfe732-48c4-48a5-86d2-9d07020b8a86 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:29.600478 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfe732-48c4-48a5-86d2-9d07020b8a86 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:29.600987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfe732-48c4-48a5-86d2-9d07020b8a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:29.635111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-62dfe732-48c4-48a5-86d2-9d07020b8a86 None None] MFA Rules not processed for user `5618db388e674db4b2e35c7da6dc6911`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:29.636571 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-62dfe732-48c4-48a5-86d2-9d07020b8a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:29.648418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308ca29-0c55-4b1f-9bb5-c5046e4cf896 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:29.649087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308ca29-0c55-4b1f-9bb5-c5046e4cf896 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:29.649361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308ca29-0c55-4b1f-9bb5-c5046e4cf896 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:29.649630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308ca29-0c55-4b1f-9bb5-c5046e4cf896 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:29.659102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308ca29-0c55-4b1f-9bb5-c5046e4cf896 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:29.660774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308ca29-0c55-4b1f-9bb5-c5046e4cf896 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:29.671909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 897/1815] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:29.673217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b40c7-1712-4643-93c2-c5b849cef810 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:29.673492 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b40c7-1712-4643-93c2-c5b849cef810 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:29.673762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b40c7-1712-4643-93c2-c5b849cef810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:29.702976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:29.707408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-da6b40c7-1712-4643-93c2-c5b849cef810 None None] MFA Rules not processed for user `f669fdee32704583bfb471651f74226a`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:29.710728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-da6b40c7-1712-4643-93c2-c5b849cef810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:29.732621 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 919/1816] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:29.737171 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfe732-48c4-48a5-86d2-9d07020b8a86 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:29.828000 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 898/1817] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:29.841608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfe732-48c4-48a5-86d2-9d07020b8a86 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:29.843558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b40c7-1712-4643-93c2-c5b849cef810 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:29.847590 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11da94c-a9da-4f7a-bd21-08e6d451d7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:29.848202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11da94c-a9da-4f7a-bd21-08e6d451d7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:29.848475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11da94c-a9da-4f7a-bd21-08e6d451d7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:29.849689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11da94c-a9da-4f7a-bd21-08e6d451d7a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:29.849689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11da94c-a9da-4f7a-bd21-08e6d451d7a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:29.851745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11da94c-a9da-4f7a-bd21-08e6d451d7a3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:29.899605 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 920/1818] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:29 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:24:29.903330 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11da94c-a9da-4f7a-bd21-08e6d451d7a3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:29.933541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b40c7-1712-4643-93c2-c5b849cef810 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:29.998507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:30.003274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94381e3-7d4b-4557-8bee-5d60be827714 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:30.004734 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94381e3-7d4b-4557-8bee-5d60be827714 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:30.005619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94381e3-7d4b-4557-8bee-5d60be827714 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:30.005952 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94381e3-7d4b-4557-8bee-5d60be827714 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:30.010100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11da94c-a9da-4f7a-bd21-08e6d451d7a3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:30.034381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fcdce-5762-4386-9ae9-3fec09645ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:30.035068 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fcdce-5762-4386-9ae9-3fec09645ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:30.036027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fcdce-5762-4386-9ae9-3fec09645ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:30.036287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fcdce-5762-4386-9ae9-3fec09645ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:30.036937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fcdce-5762-4386-9ae9-3fec09645ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:30.042280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fcdce-5762-4386-9ae9-3fec09645ea2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:30.087356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94381e3-7d4b-4557-8bee-5d60be827714 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:30.090237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94381e3-7d4b-4557-8bee-5d60be827714 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:30.093617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 921/1819] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:29 2022] POST /identity/v3/users => generated 300 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:24:30.098267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fcdce-5762-4386-9ae9-3fec09645ea2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:30.109600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fcdce-5762-4386-9ae9-3fec09645ea2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:30.115561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f759892-6269-4afb-b672-8cf17d2b270e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:30.116375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f759892-6269-4afb-b672-8cf17d2b270e admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:30.116905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f759892-6269-4afb-b672-8cf17d2b270e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:30.117392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f759892-6269-4afb-b672-8cf17d2b270e admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:30.118253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f759892-6269-4afb-b672-8cf17d2b270e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:30.119035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 899/1820] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:24:30.120442 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f759892-6269-4afb-b672-8cf17d2b270e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:30.139325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 922/1821] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:24:30.143284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94381e3-7d4b-4557-8bee-5d60be827714 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:30.150397 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:30.156851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94381e3-7d4b-4557-8bee-5d60be827714 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:30.164260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a73e09-ef0c-4498-ab74-2af8fca0b4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:30.165493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a73e09-ef0c-4498-ab74-2af8fca0b4bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:30.165862 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a73e09-ef0c-4498-ab74-2af8fca0b4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:30.166244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a73e09-ef0c-4498-ab74-2af8fca0b4bf admin admin] PATH_INFO: `/v3/projects/30fcd5eb5a2c4c1aba2e53471de07510/users/1c50c3817e1046cf9964cc95984740c1/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:30.182965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a73e09-ef0c-4498-ab74-2af8fca0b4bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=30fcd5eb5a2c4c1aba2e53471de07510, user_id=1c50c3817e1046cf9964cc95984740c1, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:30.186409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a73e09-ef0c-4498-ab74-2af8fca0b4bf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:30.203654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 900/1822] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:30 2022] PUT /identity/v3/projects/30fcd5eb5a2c4c1aba2e53471de07510/users/1c50c3817e1046cf9964cc95984740c1/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:24:30.208940 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f759892-6269-4afb-b672-8cf17d2b270e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:30.221194 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f759892-6269-4afb-b672-8cf17d2b270e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:30.279768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a73e09-ef0c-4498-ab74-2af8fca0b4bf admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:30.281493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-20a73e09-ef0c-4498-ab74-2af8fca0b4bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:30.298768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef606f-142e-4a80-8909-c311c69e15ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:30.299378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef606f-142e-4a80-8909-c311c69e15ac admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:30.299654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef606f-142e-4a80-8909-c311c69e15ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:30.299924 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef606f-142e-4a80-8909-c311c69e15ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:30.300572 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef606f-142e-4a80-8909-c311c69e15ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:30.302650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef606f-142e-4a80-8909-c311c69e15ac admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:30.314909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 923/1823] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:30.319040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef606f-142e-4a80-8909-c311c69e15ac admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:30.329295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef606f-142e-4a80-8909-c311c69e15ac admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:30.334207 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3c148-8378-4f61-80e2-51332c07f609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:30.334854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3c148-8378-4f61-80e2-51332c07f609 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:30.335132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3c148-8378-4f61-80e2-51332c07f609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:30.335403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3c148-8378-4f61-80e2-51332c07f609 admin admin] PATH_INFO: `/v3/projects/30fcd5eb5a2c4c1aba2e53471de07510/users/1c50c3817e1046cf9964cc95984740c1/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:30.338012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3c148-8378-4f61-80e2-51332c07f609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30fcd5eb5a2c4c1aba2e53471de07510, user_id=1c50c3817e1046cf9964cc95984740c1, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:30.340595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a73e09-ef0c-4498-ab74-2af8fca0b4bf admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:30.340791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3c148-8378-4f61-80e2-51332c07f609 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:30.344333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0d78d-b7de-43fc-8fbb-a760d54abde0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:30.344930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0d78d-b7de-43fc-8fbb-a760d54abde0 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:30.345205 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0d78d-b7de-43fc-8fbb-a760d54abde0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:30.345477 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0d78d-b7de-43fc-8fbb-a760d54abde0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:30.351011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 924/1824] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:30 2022] PUT /identity/v3/projects/30fcd5eb5a2c4c1aba2e53471de07510/users/1c50c3817e1046cf9964cc95984740c1/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:24:30.361632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04ccc7-d780-4ec6-9fde-92416223e781 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:30.361924 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04ccc7-d780-4ec6-9fde-92416223e781 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:30.362245 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04ccc7-d780-4ec6-9fde-92416223e781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:30.410391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6e04ccc7-d780-4ec6-9fde-92416223e781 None None] MFA Rules not processed for user `1c50c3817e1046cf9964cc95984740c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:30.411884 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6e04ccc7-d780-4ec6-9fde-92416223e781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:30.473559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0d78d-b7de-43fc-8fbb-a760d54abde0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:30.475636 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0d78d-b7de-43fc-8fbb-a760d54abde0 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:30.494784 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 901/1825] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:30.526201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 925/1826] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:24:30.533961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0d78d-b7de-43fc-8fbb-a760d54abde0 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:30.543050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0d78d-b7de-43fc-8fbb-a760d54abde0 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:30.667594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ab3a0-41f3-4b1c-bef3-0c974ac6631d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:30.667594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ab3a0-41f3-4b1c-bef3-0c974ac6631d service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:30.669570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ab3a0-41f3-4b1c-bef3-0c974ac6631d service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:30.669570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ab3a0-41f3-4b1c-bef3-0c974ac6631d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:30.704562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ab3a0-41f3-4b1c-bef3-0c974ac6631d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:30.707305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ab3a0-41f3-4b1c-bef3-0c974ac6631d service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:30.729558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 902/1827] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:30.909788 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04ccc7-d780-4ec6-9fde-92416223e781 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:30.920991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04ccc7-d780-4ec6-9fde-92416223e781 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:30.926390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72506c-10a6-4e8f-89d9-ea2a79fd1fc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:30.927369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72506c-10a6-4e8f-89d9-ea2a79fd1fc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:30.927803 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72506c-10a6-4e8f-89d9-ea2a79fd1fc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:30.928228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72506c-10a6-4e8f-89d9-ea2a79fd1fc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:30.933234 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5b72506c-10a6-4e8f-89d9-ea2a79fd1fc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:31.023862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:31.053831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72506c-10a6-4e8f-89d9-ea2a79fd1fc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:31.055838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72506c-10a6-4e8f-89d9-ea2a79fd1fc6 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:31.074276 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 926/1828] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:31.954456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ab3a0-41f3-4b1c-bef3-0c974ac6631d service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:32.057866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:32.103361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ab3a0-41f3-4b1c-bef3-0c974ac6631d service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:32.123818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:32.124477 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:32.124835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:32.125121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] PATH_INFO: `/v3/users/5618db388e674db4b2e35c7da6dc6911` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:32.126308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5618db388e674db4b2e35c7da6dc6911)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:32.128328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:32.195240 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5618db388e674db4b2e35c7da6dc6911'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:32.196299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5618db388e674db4b2e35c7da6dc6911'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:32.205747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5618db388e674db4b2e35c7da6dc6911'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:32.215503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5618db388e674db4b2e35c7da6dc6911'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:32.252779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5618db388e674db4b2e35c7da6dc6911'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:32.284773 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 903/1829] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:31 2022] DELETE /identity/v3/users/5618db388e674db4b2e35c7da6dc6911 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:32.289277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868cb59-064c-4010-89ac-1e2607452b5d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:32.289823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868cb59-064c-4010-89ac-1e2607452b5d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:32.289931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868cb59-064c-4010-89ac-1e2607452b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:32.316493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c868cb59-064c-4010-89ac-1e2607452b5d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:32.317800 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c868cb59-064c-4010-89ac-1e2607452b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:32.409932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 927/1830] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:32.423367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:32.424961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:32.515320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba89b77-6721-46bb-86fc-06ec0d87909e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:32.519550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ad945-518b-4750-b1e9-f0073475bdda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:32.520098 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ad945-518b-4750-b1e9-f0073475bdda service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:32.521253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ad945-518b-4750-b1e9-f0073475bdda service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:32.521534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ad945-518b-4750-b1e9-f0073475bdda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:32.541657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ad945-518b-4750-b1e9-f0073475bdda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:32.544674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ad945-518b-4750-b1e9-f0073475bdda service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:32.558834 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 904/1831] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:32.972286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868cb59-064c-4010-89ac-1e2607452b5d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:32.973745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c868cb59-064c-4010-89ac-1e2607452b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:32.994823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868cb59-064c-4010-89ac-1e2607452b5d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:32.999969 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:33.000643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:33.000971 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:33.001151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] PATH_INFO: `/v3/projects/3267e0059119481e81c82606a7a304ed` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:33.002360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3267e0059119481e81c82606a7a304ed)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:33.004407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:33.061478 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3267e0059119481e81c82606a7a304ed', 'request_id': 'req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.067825 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3267e0059119481e81c82606a7a304ed', 'request_id': 'req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.076151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3267e0059119481e81c82606a7a304ed', 'request_id': 'req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.091017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] The token cache is being invalidate because project 3267e0059119481e81c82606a7a304ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:33.091124 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.092062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.097064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 928/1832] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:32 2022] DELETE /identity/v3/projects/3267e0059119481e81c82606a7a304ed => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:33.098412 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ad945-518b-4750-b1e9-f0073475bdda service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:33.109879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ad945-518b-4750-b1e9-f0073475bdda service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:33.186707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557051a-5a9f-4f0c-829f-8f555c00fa58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:33.187329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557051a-5a9f-4f0c-829f-8f555c00fa58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:33.187601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557051a-5a9f-4f0c-829f-8f555c00fa58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:33.187869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557051a-5a9f-4f0c-829f-8f555c00fa58 admin admin] PATH_INFO: `/v3/users/1c50c3817e1046cf9964cc95984740c1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:33.188977 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557051a-5a9f-4f0c-829f-8f555c00fa58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c50c3817e1046cf9964cc95984740c1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:33.190921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557051a-5a9f-4f0c-829f-8f555c00fa58 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:33.192393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:33.242649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3557051a-5a9f-4f0c-829f-8f555c00fa58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c50c3817e1046cf9964cc95984740c1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.243591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3557051a-5a9f-4f0c-829f-8f555c00fa58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c50c3817e1046cf9964cc95984740c1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.251598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3557051a-5a9f-4f0c-829f-8f555c00fa58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c50c3817e1046cf9964cc95984740c1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.260276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3557051a-5a9f-4f0c-829f-8f555c00fa58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c50c3817e1046cf9964cc95984740c1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.280895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3557051a-5a9f-4f0c-829f-8f555c00fa58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c50c3817e1046cf9964cc95984740c1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.304633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 905/1833] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:33 2022] DELETE /identity/v3/users/1c50c3817e1046cf9964cc95984740c1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:33.649314 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:33.650877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:33.664042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b4fe4b-93d9-42ad-b6d8-456a8ab4dbb0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:33.730829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc65a4-d1b8-459b-a852-bb3eadd440a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:33.731463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc65a4-d1b8-459b-a852-bb3eadd440a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:33.731741 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc65a4-d1b8-459b-a852-bb3eadd440a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:33.732018 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc65a4-d1b8-459b-a852-bb3eadd440a7 admin admin] PATH_INFO: `/v3/projects/30fcd5eb5a2c4c1aba2e53471de07510` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:33.733150 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc65a4-d1b8-459b-a852-bb3eadd440a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30fcd5eb5a2c4c1aba2e53471de07510)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:33.735097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc65a4-d1b8-459b-a852-bb3eadd440a7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:33.737743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:33.785990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4efc65a4-d1b8-459b-a852-bb3eadd440a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30fcd5eb5a2c4c1aba2e53471de07510', 'request_id': 'req-4efc65a4-d1b8-459b-a852-bb3eadd440a7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.791532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4efc65a4-d1b8-459b-a852-bb3eadd440a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30fcd5eb5a2c4c1aba2e53471de07510', 'request_id': 'req-4efc65a4-d1b8-459b-a852-bb3eadd440a7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.798277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4efc65a4-d1b8-459b-a852-bb3eadd440a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30fcd5eb5a2c4c1aba2e53471de07510', 'request_id': 'req-4efc65a4-d1b8-459b-a852-bb3eadd440a7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.810056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4efc65a4-d1b8-459b-a852-bb3eadd440a7 admin admin] The token cache is being invalidate because project 30fcd5eb5a2c4c1aba2e53471de07510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:33.810368 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4efc65a4-d1b8-459b-a852-bb3eadd440a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.811338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4efc65a4-d1b8-459b-a852-bb3eadd440a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:33.812750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 929/1834] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:33 2022] DELETE /identity/v3/projects/30fcd5eb5a2c4c1aba2e53471de07510 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:33.818401 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa93564-c6c6-411d-834b-e7b0821af261 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:33.818692 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa93564-c6c6-411d-834b-e7b0821af261 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:33.818966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa93564-c6c6-411d-834b-e7b0821af261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:33.838134 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6fa93564-c6c6-411d-834b-e7b0821af261 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:33.839305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6fa93564-c6c6-411d-834b-e7b0821af261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:33.898747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 906/1835] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:33.906824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a9a4-b1de-4dcf-9bf6-67d04ca41799 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:33.907101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a9a4-b1de-4dcf-9bf6-67d04ca41799 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:33.907367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533a9a4-b1de-4dcf-9bf6-67d04ca41799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:33.928185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7533a9a4-b1de-4dcf-9bf6-67d04ca41799 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:33.929366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7533a9a4-b1de-4dcf-9bf6-67d04ca41799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:33.937386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 930/1836] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:33.941441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa93564-c6c6-411d-834b-e7b0821af261 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:33.950986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa93564-c6c6-411d-834b-e7b0821af261 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:33.955456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f371ef4-490d-4151-b5c0-c62ae4b3a4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:33.956036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f371ef4-490d-4151-b5c0-c62ae4b3a4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:33.956306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f371ef4-490d-4151-b5c0-c62ae4b3a4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:33.956570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f371ef4-490d-4151-b5c0-c62ae4b3a4c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:33.957204 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f371ef4-490d-4151-b5c0-c62ae4b3a4c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:33.957523 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f371ef4-490d-4151-b5c0-c62ae4b3a4c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:33.959120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f371ef4-490d-4151-b5c0-c62ae4b3a4c2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:33.970815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 907/1837] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:24:33.974979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533a9a4-b1de-4dcf-9bf6-67d04ca41799 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:33.984923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533a9a4-b1de-4dcf-9bf6-67d04ca41799 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:33.989686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7d704-f8a3-4605-b199-5ee4e0f8b853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:33.990416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7d704-f8a3-4605-b199-5ee4e0f8b853 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:33.990693 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7d704-f8a3-4605-b199-5ee4e0f8b853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:33.990962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7d704-f8a3-4605-b199-5ee4e0f8b853 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:33.991606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7d704-f8a3-4605-b199-5ee4e0f8b853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:33.993577 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7d704-f8a3-4605-b199-5ee4e0f8b853 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:34.021039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 931/1838] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:33 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:34.025165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f371ef4-490d-4151-b5c0-c62ae4b3a4c2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:34.035410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f371ef4-490d-4151-b5c0-c62ae4b3a4c2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:34.089560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4df125-39d1-4855-a881-6ad333ee6be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:34.090169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4df125-39d1-4855-a881-6ad333ee6be8 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:34.090443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4df125-39d1-4855-a881-6ad333ee6be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:34.090707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4df125-39d1-4855-a881-6ad333ee6be8 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:34.091322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4df125-39d1-4855-a881-6ad333ee6be8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:34.093203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4df125-39d1-4855-a881-6ad333ee6be8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:34.103875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:34.117132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 908/1839] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:34 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:34.121191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7d704-f8a3-4605-b199-5ee4e0f8b853 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:34.131853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7d704-f8a3-4605-b199-5ee4e0f8b853 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:34.136410 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404e22e-7a1c-4663-8129-fecea69f9671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:34.137068 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404e22e-7a1c-4663-8129-fecea69f9671 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:34.137344 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404e22e-7a1c-4663-8129-fecea69f9671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:34.137613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404e22e-7a1c-4663-8129-fecea69f9671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:34.138287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404e22e-7a1c-4663-8129-fecea69f9671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:34.139934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404e22e-7a1c-4663-8129-fecea69f9671 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:34.150025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 932/1840] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:34.153697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4df125-39d1-4855-a881-6ad333ee6be8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:34.163328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4df125-39d1-4855-a881-6ad333ee6be8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:34.167822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4834c4-8521-4424-adff-b7334fb387f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:34.168903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4834c4-8521-4424-adff-b7334fb387f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:34.169179 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4834c4-8521-4424-adff-b7334fb387f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:34.169450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4834c4-8521-4424-adff-b7334fb387f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:34.170099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4834c4-8521-4424-adff-b7334fb387f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:34.171712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4834c4-8521-4424-adff-b7334fb387f7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:34.182020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 909/1841] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:34.186347 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404e22e-7a1c-4663-8129-fecea69f9671 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:34.195930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404e22e-7a1c-4663-8129-fecea69f9671 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:34.200477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a997f6-a5b9-4f42-8997-0310f9238236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:34.201117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a997f6-a5b9-4f42-8997-0310f9238236 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:34.201391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a997f6-a5b9-4f42-8997-0310f9238236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:34.201658 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a997f6-a5b9-4f42-8997-0310f9238236 admin admin] PATH_INFO: `/v3/projects/3bcd8d3a7a554a759c0bee29b027e2df/users/6d523d374b30439e8d84080a749b70d0/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:34.215959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a997f6-a5b9-4f42-8997-0310f9238236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bcd8d3a7a554a759c0bee29b027e2df, user_id=6d523d374b30439e8d84080a749b70d0, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:34.218657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a997f6-a5b9-4f42-8997-0310f9238236 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:34.232573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 933/1842] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:34 2022] PUT /identity/v3/projects/3bcd8d3a7a554a759c0bee29b027e2df/users/6d523d374b30439e8d84080a749b70d0/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:34.244389 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f778a70-e4b9-4cda-9805-4f68a3f7f89e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:34.244675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f778a70-e4b9-4cda-9805-4f68a3f7f89e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:34.244950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f778a70-e4b9-4cda-9805-4f68a3f7f89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:34.278178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9f778a70-e4b9-4cda-9805-4f68a3f7f89e None None] MFA Rules not processed for user `6d523d374b30439e8d84080a749b70d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:34.279420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9f778a70-e4b9-4cda-9805-4f68a3f7f89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:34.351673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 910/1843] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:34.356545 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a997f6-a5b9-4f42-8997-0310f9238236 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:34.424334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a997f6-a5b9-4f42-8997-0310f9238236 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:34.429428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce865d5c-dd75-4e33-9120-ab167c2ed7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:34.430183 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce865d5c-dd75-4e33-9120-ab167c2ed7fa admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:34.430562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce865d5c-dd75-4e33-9120-ab167c2ed7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:34.431009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce865d5c-dd75-4e33-9120-ab167c2ed7fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:34.431700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce865d5c-dd75-4e33-9120-ab167c2ed7fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:34.433688 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce865d5c-dd75-4e33-9120-ab167c2ed7fa admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:34.466352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 934/1844] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:34 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:24:34.467818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f778a70-e4b9-4cda-9805-4f68a3f7f89e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:34.479073 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f778a70-e4b9-4cda-9805-4f68a3f7f89e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:34.547124 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc705ee6-5d09-42ee-948b-ae74a8551d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:34.547773 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc705ee6-5d09-42ee-948b-ae74a8551d2f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:34.548054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc705ee6-5d09-42ee-948b-ae74a8551d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:34.548321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc705ee6-5d09-42ee-948b-ae74a8551d2f admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:34.548970 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc705ee6-5d09-42ee-948b-ae74a8551d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:34.550903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc705ee6-5d09-42ee-948b-ae74a8551d2f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:34.587077 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 911/1845] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:34 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:24:34.589339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce865d5c-dd75-4e33-9120-ab167c2ed7fa admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:34.600542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce865d5c-dd75-4e33-9120-ab167c2ed7fa admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:34.607446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c608fb9-bc3b-4f43-b861-7d0aff5add6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:34.608061 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c608fb9-bc3b-4f43-b861-7d0aff5add6f admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:34.608339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c608fb9-bc3b-4f43-b861-7d0aff5add6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:34.608691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c608fb9-bc3b-4f43-b861-7d0aff5add6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:34.609469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c608fb9-bc3b-4f43-b861-7d0aff5add6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:34.610910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c608fb9-bc3b-4f43-b861-7d0aff5add6f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:34.624346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 935/1846] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:34.628795 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc705ee6-5d09-42ee-948b-ae74a8551d2f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:34.640370 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc705ee6-5d09-42ee-948b-ae74a8551d2f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:34.645884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187be74-bd83-4381-acd8-dd0f615af079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:34.646772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187be74-bd83-4381-acd8-dd0f615af079 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:34.647064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187be74-bd83-4381-acd8-dd0f615af079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:34.647339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187be74-bd83-4381-acd8-dd0f615af079 admin admin] PATH_INFO: `/v3/projects/d484094be5bd4105b6764a62581c3d5b/users/b1a3b37531024d37a5801e21327cd467/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:34.666625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187be74-bd83-4381-acd8-dd0f615af079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d484094be5bd4105b6764a62581c3d5b, user_id=b1a3b37531024d37a5801e21327cd467, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:34.669344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0187be74-bd83-4381-acd8-dd0f615af079 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:34.686289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 912/1847] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:34 2022] PUT /identity/v3/projects/d484094be5bd4105b6764a62581c3d5b/users/b1a3b37531024d37a5801e21327cd467/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:34.690765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c608fb9-bc3b-4f43-b861-7d0aff5add6f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:34.695919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:34.704192 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c608fb9-bc3b-4f43-b861-7d0aff5add6f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:34.791582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac84b9-56a6-47ac-a7b7-1bc77e273f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:34.792373 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac84b9-56a6-47ac-a7b7-1bc77e273f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:34.792922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac84b9-56a6-47ac-a7b7-1bc77e273f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:34.793085 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac84b9-56a6-47ac-a7b7-1bc77e273f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:34.793796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac84b9-56a6-47ac-a7b7-1bc77e273f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:34.795609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac84b9-56a6-47ac-a7b7-1bc77e273f81 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:34.812807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 936/1848] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:34 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:34.816924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187be74-bd83-4381-acd8-dd0f615af079 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:34.841726 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187be74-bd83-4381-acd8-dd0f615af079 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:34.853339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de71aa1-3e81-4837-8848-e795326e6ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:34.854040 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de71aa1-3e81-4837-8848-e795326e6ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:34.854312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de71aa1-3e81-4837-8848-e795326e6ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:34.854584 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de71aa1-3e81-4837-8848-e795326e6ea3 admin admin] PATH_INFO: `/v3/projects/d484094be5bd4105b6764a62581c3d5b/users/b1a3b37531024d37a5801e21327cd467/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:34.857365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de71aa1-3e81-4837-8848-e795326e6ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d484094be5bd4105b6764a62581c3d5b, user_id=b1a3b37531024d37a5801e21327cd467, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:34.860230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de71aa1-3e81-4837-8848-e795326e6ea3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:34.885236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 913/1849] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:34 2022] PUT /identity/v3/projects/d484094be5bd4105b6764a62581c3d5b/users/b1a3b37531024d37a5801e21327cd467/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:34.895920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb1d49-4313-4ee1-876d-292c2bea1018 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:34.896206 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb1d49-4313-4ee1-876d-292c2bea1018 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:34.896476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb1d49-4313-4ee1-876d-292c2bea1018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:34.935324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1bfb1d49-4313-4ee1-876d-292c2bea1018 None None] MFA Rules not processed for user `b1a3b37531024d37a5801e21327cd467`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:34.936597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1bfb1d49-4313-4ee1-876d-292c2bea1018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:35.012999 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 937/1850] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:35.025416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de71aa1-3e81-4837-8848-e795326e6ea3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:35.027211 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5de71aa1-3e81-4837-8848-e795326e6ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:35.059999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:35.088459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de71aa1-3e81-4837-8848-e795326e6ea3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:35.092228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7d451-197a-4dfd-9a25-add720480f8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:35.092928 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7d451-197a-4dfd-9a25-add720480f8f service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:35.093203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7d451-197a-4dfd-9a25-add720480f8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:35.093468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7d451-197a-4dfd-9a25-add720480f8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:35.103412 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7d451-197a-4dfd-9a25-add720480f8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:35.104766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7d451-197a-4dfd-9a25-add720480f8f service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:35.115514 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 914/1851] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:35.210718 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb1d49-4313-4ee1-876d-292c2bea1018 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:35.220619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb1d49-4313-4ee1-876d-292c2bea1018 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:35.224285 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e390504-6485-4ef1-8e5c-e0c9c4620f9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:35.224950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e390504-6485-4ef1-8e5c-e0c9c4620f9c service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:35.225220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e390504-6485-4ef1-8e5c-e0c9c4620f9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:35.225482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e390504-6485-4ef1-8e5c-e0c9c4620f9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:35.286406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e390504-6485-4ef1-8e5c-e0c9c4620f9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:35.290575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e390504-6485-4ef1-8e5c-e0c9c4620f9c service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:35.302968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 938/1852] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:35.622159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7d451-197a-4dfd-9a25-add720480f8f service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:35.710672 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7d451-197a-4dfd-9a25-add720480f8f service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:35.715333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:35.715961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:35.716236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:35.716509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] PATH_INFO: `/v3/users/6d523d374b30439e8d84080a749b70d0` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:35.717681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d523d374b30439e8d84080a749b70d0)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:35.719636 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:35.775908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d523d374b30439e8d84080a749b70d0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:35.776600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d523d374b30439e8d84080a749b70d0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:35.785246 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d523d374b30439e8d84080a749b70d0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:35.793914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d523d374b30439e8d84080a749b70d0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:35.825154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d523d374b30439e8d84080a749b70d0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:35.831848 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 915/1853] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:35 2022] DELETE /identity/v3/users/6d523d374b30439e8d84080a749b70d0 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:35.836285 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0627b25-05d5-473b-93c5-0e10e7774c19 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:35.836816 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0627b25-05d5-473b-93c5-0e10e7774c19 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:35.836928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0627b25-05d5-473b-93c5-0e10e7774c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:35.857777 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d0627b25-05d5-473b-93c5-0e10e7774c19 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:35.859315 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d0627b25-05d5-473b-93c5-0e10e7774c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:35.878041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:35.927749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 939/1854] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:35.939310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:35.940805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:36.003507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:36.008462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1aaa6-918f-44ee-b3c4-e02b0fceb0f0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:36.014242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ec5b7-09ee-4a8b-86db-9d1515b63551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:36.015138 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ec5b7-09ee-4a8b-86db-9d1515b63551 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:36.015574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ec5b7-09ee-4a8b-86db-9d1515b63551 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:36.015981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ec5b7-09ee-4a8b-86db-9d1515b63551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:36.028899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ec5b7-09ee-4a8b-86db-9d1515b63551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:36.030922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ec5b7-09ee-4a8b-86db-9d1515b63551 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:36.048521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 916/1855] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:36.364166 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0627b25-05d5-473b-93c5-0e10e7774c19 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:36.366108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d0627b25-05d5-473b-93c5-0e10e7774c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:36.379607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0627b25-05d5-473b-93c5-0e10e7774c19 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:36.384776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:36.385572 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:36.386014 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:36.386614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] PATH_INFO: `/v3/projects/3bcd8d3a7a554a759c0bee29b027e2df` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:36.388648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bcd8d3a7a554a759c0bee29b027e2df)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:36.390766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:36.440714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bcd8d3a7a554a759c0bee29b027e2df', 'request_id': 'req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:36.446507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bcd8d3a7a554a759c0bee29b027e2df', 'request_id': 'req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:36.453652 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bcd8d3a7a554a759c0bee29b027e2df', 'request_id': 'req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:36.466911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] The token cache is being invalidate because project 3bcd8d3a7a554a759c0bee29b027e2df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:36.467197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:36.467829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:36.469322 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 940/1856] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:36 2022] DELETE /identity/v3/projects/3bcd8d3a7a554a759c0bee29b027e2df => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:36.473353 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ec5b7-09ee-4a8b-86db-9d1515b63551 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:36.483106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ec5b7-09ee-4a8b-86db-9d1515b63551 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:36.554338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f54c90-ce46-421c-9547-5e0da87f2583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:36.555173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f54c90-ce46-421c-9547-5e0da87f2583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:36.555552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f54c90-ce46-421c-9547-5e0da87f2583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:36.555932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f54c90-ce46-421c-9547-5e0da87f2583 admin admin] PATH_INFO: `/v3/users/b1a3b37531024d37a5801e21327cd467` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:36.557297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f54c90-ce46-421c-9547-5e0da87f2583 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a3b37531024d37a5801e21327cd467)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:36.559849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f54c90-ce46-421c-9547-5e0da87f2583 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:36.639618 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-46f54c90-ce46-421c-9547-5e0da87f2583 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a3b37531024d37a5801e21327cd467'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:36.640332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-46f54c90-ce46-421c-9547-5e0da87f2583 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a3b37531024d37a5801e21327cd467'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:36.648012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-46f54c90-ce46-421c-9547-5e0da87f2583 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a3b37531024d37a5801e21327cd467'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:36.662641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-46f54c90-ce46-421c-9547-5e0da87f2583 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a3b37531024d37a5801e21327cd467'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:36.676730 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-46f54c90-ce46-421c-9547-5e0da87f2583 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a3b37531024d37a5801e21327cd467'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:36.688280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 917/1857] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:36 2022] DELETE /identity/v3/users/b1a3b37531024d37a5801e21327cd467 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:37.010139 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:37.011722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:37.034027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e73e1-b3fc-4baf-89f4-82537b1e06a8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:37.122949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614071bd-076f-4abf-8d70-42b4c203ed7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:37.124668 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614071bd-076f-4abf-8d70-42b4c203ed7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:37.124903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614071bd-076f-4abf-8d70-42b4c203ed7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:37.125183 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614071bd-076f-4abf-8d70-42b4c203ed7e admin admin] PATH_INFO: `/v3/projects/d484094be5bd4105b6764a62581c3d5b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:37.126652 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614071bd-076f-4abf-8d70-42b4c203ed7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d484094be5bd4105b6764a62581c3d5b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:37.129069 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614071bd-076f-4abf-8d70-42b4c203ed7e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:37.253731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:37.256627 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-614071bd-076f-4abf-8d70-42b4c203ed7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd484094be5bd4105b6764a62581c3d5b', 'request_id': 'req-614071bd-076f-4abf-8d70-42b4c203ed7e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:37.265905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-614071bd-076f-4abf-8d70-42b4c203ed7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd484094be5bd4105b6764a62581c3d5b', 'request_id': 'req-614071bd-076f-4abf-8d70-42b4c203ed7e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:37.282956 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-614071bd-076f-4abf-8d70-42b4c203ed7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd484094be5bd4105b6764a62581c3d5b', 'request_id': 'req-614071bd-076f-4abf-8d70-42b4c203ed7e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:37.300211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-614071bd-076f-4abf-8d70-42b4c203ed7e admin admin] The token cache is being invalidate because project d484094be5bd4105b6764a62581c3d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:37.300732 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-614071bd-076f-4abf-8d70-42b4c203ed7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:37.301569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-614071bd-076f-4abf-8d70-42b4c203ed7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:37.303987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 941/1858] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:37 2022] DELETE /identity/v3/projects/d484094be5bd4105b6764a62581c3d5b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:24:37.310108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba7745-4ebc-431a-9fea-da7699f6d27c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:37.310377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba7745-4ebc-431a-9fea-da7699f6d27c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:37.310657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba7745-4ebc-431a-9fea-da7699f6d27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:37.350317 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-11ba7745-4ebc-431a-9fea-da7699f6d27c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:37.352745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-11ba7745-4ebc-431a-9fea-da7699f6d27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:37.435007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 918/1859] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:37.442743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d417a3-6ba2-48c5-b9d0-529cdb3a10de None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:37.443235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d417a3-6ba2-48c5-b9d0-529cdb3a10de None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:37.443580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d417a3-6ba2-48c5-b9d0-529cdb3a10de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:37.464884 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-66d417a3-6ba2-48c5-b9d0-529cdb3a10de None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:37.466276 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-66d417a3-6ba2-48c5-b9d0-529cdb3a10de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:37.474828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 942/1860] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:37.478962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba7745-4ebc-431a-9fea-da7699f6d27c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:37.489520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba7745-4ebc-431a-9fea-da7699f6d27c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:37.494636 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c9dc8-44bb-4565-9f0c-5b8c3b55a975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:37.495236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c9dc8-44bb-4565-9f0c-5b8c3b55a975 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:37.495511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c9dc8-44bb-4565-9f0c-5b8c3b55a975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:37.495777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c9dc8-44bb-4565-9f0c-5b8c3b55a975 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:37.496981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c9dc8-44bb-4565-9f0c-5b8c3b55a975 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:37.497280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c9dc8-44bb-4565-9f0c-5b8c3b55a975 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:37.498956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c9dc8-44bb-4565-9f0c-5b8c3b55a975 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:37.505875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:37.511247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 919/1861] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:37.516565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d417a3-6ba2-48c5-b9d0-529cdb3a10de None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:37.525618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d417a3-6ba2-48c5-b9d0-529cdb3a10de None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:37.530872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529825e9-f7ea-4d23-950d-2b9f74e91560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:37.531469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529825e9-f7ea-4d23-950d-2b9f74e91560 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:37.531744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529825e9-f7ea-4d23-950d-2b9f74e91560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:37.532013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529825e9-f7ea-4d23-950d-2b9f74e91560 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:37.532649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529825e9-f7ea-4d23-950d-2b9f74e91560 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:37.534577 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529825e9-f7ea-4d23-950d-2b9f74e91560 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:37.562392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 943/1862] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:37 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:37.566577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c9dc8-44bb-4565-9f0c-5b8c3b55a975 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:37.576579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c9dc8-44bb-4565-9f0c-5b8c3b55a975 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:37.634026 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb19d29-7f10-4175-bc83-35ae5dfd9b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:37.634673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb19d29-7f10-4175-bc83-35ae5dfd9b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:37.634954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb19d29-7f10-4175-bc83-35ae5dfd9b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:37.635223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb19d29-7f10-4175-bc83-35ae5dfd9b53 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:37.635893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb19d29-7f10-4175-bc83-35ae5dfd9b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:37.637801 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb19d29-7f10-4175-bc83-35ae5dfd9b53 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:37.659674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 920/1863] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:37 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:37.663725 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529825e9-f7ea-4d23-950d-2b9f74e91560 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:37.674640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529825e9-f7ea-4d23-950d-2b9f74e91560 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:37.679197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c63c2-670d-41ff-a671-52c1572a13ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:37.679788 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c63c2-670d-41ff-a671-52c1572a13ce admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:37.680065 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c63c2-670d-41ff-a671-52c1572a13ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:37.680336 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c63c2-670d-41ff-a671-52c1572a13ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:37.680981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c63c2-670d-41ff-a671-52c1572a13ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:37.682624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c63c2-670d-41ff-a671-52c1572a13ce admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:37.692457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 944/1864] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:37.696205 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb19d29-7f10-4175-bc83-35ae5dfd9b53 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:37.706191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb19d29-7f10-4175-bc83-35ae5dfd9b53 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:37.710931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d28c38-cedc-4034-b79d-e61cd6b22496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:37.711517 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28c38-cedc-4034-b79d-e61cd6b22496 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:37.711790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28c38-cedc-4034-b79d-e61cd6b22496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:37.712066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d28c38-cedc-4034-b79d-e61cd6b22496 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:37.712701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d28c38-cedc-4034-b79d-e61cd6b22496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:37.714306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d28c38-cedc-4034-b79d-e61cd6b22496 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:37.725389 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 921/1865] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:37.729310 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c63c2-670d-41ff-a671-52c1572a13ce admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:37.740743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c63c2-670d-41ff-a671-52c1572a13ce admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:37.745877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5810b-e604-4984-b949-433802e742c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:37.746742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5810b-e604-4984-b949-433802e742c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:37.747131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5810b-e604-4984-b949-433802e742c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:37.747439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5810b-e604-4984-b949-433802e742c1 admin admin] PATH_INFO: `/v3/projects/9407976c42dc41d7ad66e99b54657dbd/users/dd30b028afe14975b8b9d372f743023f/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:37.762154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5810b-e604-4984-b949-433802e742c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9407976c42dc41d7ad66e99b54657dbd, user_id=dd30b028afe14975b8b9d372f743023f, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:37.764834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5810b-e604-4984-b949-433802e742c1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:37.779080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 945/1866] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:37 2022] PUT /identity/v3/projects/9407976c42dc41d7ad66e99b54657dbd/users/dd30b028afe14975b8b9d372f743023f/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:37.790286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b6b12-9e62-4f23-bbac-91ac77e68047 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:37.790571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b6b12-9e62-4f23-bbac-91ac77e68047 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:37.790846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b6b12-9e62-4f23-bbac-91ac77e68047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:37.828640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d89b6b12-9e62-4f23-bbac-91ac77e68047 None None] MFA Rules not processed for user `dd30b028afe14975b8b9d372f743023f`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:37.830173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d89b6b12-9e62-4f23-bbac-91ac77e68047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:37.908657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 922/1867] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:37.913267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5810b-e604-4984-b949-433802e742c1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:37.976540 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5810b-e604-4984-b949-433802e742c1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:37.981400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bfe50-f2a4-4b27-b70b-30ab06b01150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:37.982008 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bfe50-f2a4-4b27-b70b-30ab06b01150 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:37.982304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bfe50-f2a4-4b27-b70b-30ab06b01150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:37.982585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bfe50-f2a4-4b27-b70b-30ab06b01150 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:37.983198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bfe50-f2a4-4b27-b70b-30ab06b01150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:37.985154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bfe50-f2a4-4b27-b70b-30ab06b01150 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:38.011058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 946/1868] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:37 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:38.016787 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b6b12-9e62-4f23-bbac-91ac77e68047 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:38.028541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b6b12-9e62-4f23-bbac-91ac77e68047 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:38.085761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfa52f-dd78-443f-a0ce-07399efc4f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:38.086503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfa52f-dd78-443f-a0ce-07399efc4f18 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:38.086840 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfa52f-dd78-443f-a0ce-07399efc4f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:38.087170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfa52f-dd78-443f-a0ce-07399efc4f18 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:38.087877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfa52f-dd78-443f-a0ce-07399efc4f18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:38.089864 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfa52f-dd78-443f-a0ce-07399efc4f18 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:38.102233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:38.144479 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 923/1869] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:38 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:38.154896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bfe50-f2a4-4b27-b70b-30ab06b01150 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:38.174247 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bfe50-f2a4-4b27-b70b-30ab06b01150 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:38.180175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82397ae1-3af2-46a3-82ca-941e6a39c3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:38.180869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82397ae1-3af2-46a3-82ca-941e6a39c3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:38.181149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82397ae1-3af2-46a3-82ca-941e6a39c3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:38.181452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82397ae1-3af2-46a3-82ca-941e6a39c3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:38.182277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82397ae1-3af2-46a3-82ca-941e6a39c3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:38.184411 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82397ae1-3af2-46a3-82ca-941e6a39c3d2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:38.195316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 947/1870] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:38.200440 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfa52f-dd78-443f-a0ce-07399efc4f18 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:38.210199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfa52f-dd78-443f-a0ce-07399efc4f18 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:38.214805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521823d-4351-409c-a09e-b1b70821cfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:38.215464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521823d-4351-409c-a09e-b1b70821cfc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:38.215752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521823d-4351-409c-a09e-b1b70821cfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:38.216016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521823d-4351-409c-a09e-b1b70821cfc5 admin admin] PATH_INFO: `/v3/projects/d64a350643054f47b50f868c27d5ad46/users/699b0306f1fe4b0f93e6982797e511e1/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:38.230682 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521823d-4351-409c-a09e-b1b70821cfc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64a350643054f47b50f868c27d5ad46, user_id=699b0306f1fe4b0f93e6982797e511e1, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:38.233257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2521823d-4351-409c-a09e-b1b70821cfc5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:38.245938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 924/1871] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:38 2022] PUT /identity/v3/projects/d64a350643054f47b50f868c27d5ad46/users/699b0306f1fe4b0f93e6982797e511e1/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:38.250434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82397ae1-3af2-46a3-82ca-941e6a39c3d2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:38.260220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82397ae1-3af2-46a3-82ca-941e6a39c3d2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:38.303635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:38.318217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d248e75-d5f3-4b45-8bae-c5a0635051b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:38.318829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d248e75-d5f3-4b45-8bae-c5a0635051b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:38.319108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d248e75-d5f3-4b45-8bae-c5a0635051b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:38.319381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d248e75-d5f3-4b45-8bae-c5a0635051b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:38.320029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d248e75-d5f3-4b45-8bae-c5a0635051b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:38.321633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d248e75-d5f3-4b45-8bae-c5a0635051b6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:38.332191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 948/1872] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:38.336647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521823d-4351-409c-a09e-b1b70821cfc5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:38.346939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521823d-4351-409c-a09e-b1b70821cfc5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:38.351490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8582ee-68ff-46c4-9a73-6a6028456e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:38.352125 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8582ee-68ff-46c4-9a73-6a6028456e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:38.352402 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8582ee-68ff-46c4-9a73-6a6028456e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:38.352672 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8582ee-68ff-46c4-9a73-6a6028456e8d admin admin] PATH_INFO: `/v3/projects/d64a350643054f47b50f868c27d5ad46/users/699b0306f1fe4b0f93e6982797e511e1/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:38.354636 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8582ee-68ff-46c4-9a73-6a6028456e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64a350643054f47b50f868c27d5ad46, user_id=699b0306f1fe4b0f93e6982797e511e1, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:38.357532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8582ee-68ff-46c4-9a73-6a6028456e8d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:38.368090 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 925/1873] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:38 2022] PUT /identity/v3/projects/d64a350643054f47b50f868c27d5ad46/users/699b0306f1fe4b0f93e6982797e511e1/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:38.378604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeb1cb-fa9c-449a-82e7-6a79c1fd510b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:38.378914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeb1cb-fa9c-449a-82e7-6a79c1fd510b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:38.379185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeeb1cb-fa9c-449a-82e7-6a79c1fd510b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:38.407585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-aaeeb1cb-fa9c-449a-82e7-6a79c1fd510b None None] MFA Rules not processed for user `699b0306f1fe4b0f93e6982797e511e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:38.408821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-aaeeb1cb-fa9c-449a-82e7-6a79c1fd510b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:38.494519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 949/1874] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:38.509425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8582ee-68ff-46c4-9a73-6a6028456e8d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:38.511377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-aa8582ee-68ff-46c4-9a73-6a6028456e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:38.576303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8582ee-68ff-46c4-9a73-6a6028456e8d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:38.580010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3415c0e-0f2a-4bbc-8869-c170b9b0159d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:38.580619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3415c0e-0f2a-4bbc-8869-c170b9b0159d service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:38.580956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3415c0e-0f2a-4bbc-8869-c170b9b0159d service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:38.581233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3415c0e-0f2a-4bbc-8869-c170b9b0159d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:38.664351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3415c0e-0f2a-4bbc-8869-c170b9b0159d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:38.665949 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3415c0e-0f2a-4bbc-8869-c170b9b0159d service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:38.713093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 926/1875] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:48.388738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5118a1b-a0f8-406b-80ca-7e760f0145d6 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:48.389707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5118a1b-a0f8-406b-80ca-7e760f0145d6 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:48.390101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5118a1b-a0f8-406b-80ca-7e760f0145d6 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:48.391502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 950/1876] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:24:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:24:48.420246 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710694ef-56cd-40a8-9c4e-acea1734f3f7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:48.420935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710694ef-56cd-40a8-9c4e-acea1734f3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:48.422366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710694ef-56cd-40a8-9c4e-acea1734f3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:48.453002 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-710694ef-56cd-40a8-9c4e-acea1734f3f7 None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:48.454407 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-710694ef-56cd-40a8-9c4e-acea1734f3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:48.491454 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=42 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:48.534546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 927/1877] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:24:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:48.550557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5118a1b-a0f8-406b-80ca-7e760f0145d6 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:48.553100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c5118a1b-a0f8-406b-80ca-7e760f0145d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:48.576563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5118a1b-a0f8-406b-80ca-7e760f0145d6 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:48.582216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3558169-f39e-4603-9405-ea5fdf6301d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:48.582822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3558169-f39e-4603-9405-ea5fdf6301d7 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:48.583274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3558169-f39e-4603-9405-ea5fdf6301d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:48.583524 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3558169-f39e-4603-9405-ea5fdf6301d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:48.592647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3558169-f39e-4603-9405-ea5fdf6301d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:48.594275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3558169-f39e-4603-9405-ea5fdf6301d7 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:48.604537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 951/1878] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:48.716846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710694ef-56cd-40a8-9c4e-acea1734f3f7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:48.718360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-710694ef-56cd-40a8-9c4e-acea1734f3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:48.724841 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:48.787092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710694ef-56cd-40a8-9c4e-acea1734f3f7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:48.791830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:48.792463 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:48.792767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:48.793038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] PATH_INFO: `/v3/users/2b212f3dd74b4def9e588be167159419` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:48.794222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b212f3dd74b4def9e588be167159419)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:48.796135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:48.851755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b212f3dd74b4def9e588be167159419'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:48.852560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b212f3dd74b4def9e588be167159419'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:48.859773 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b212f3dd74b4def9e588be167159419'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:48.868218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b212f3dd74b4def9e588be167159419'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:48.881656 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b212f3dd74b4def9e588be167159419'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:48.888364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 928/1879] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:48 2022] DELETE /identity/v3/users/2b212f3dd74b4def9e588be167159419 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:48.892912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ff6cb-08ce-4f7f-952e-0aac71090474 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:48.893239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ff6cb-08ce-4f7f-952e-0aac71090474 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:48.893514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ff6cb-08ce-4f7f-952e-0aac71090474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:48.912312 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-734ff6cb-08ce-4f7f-952e-0aac71090474 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:48.913493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-734ff6cb-08ce-4f7f-952e-0aac71090474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:48.990391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 952/1880] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:49.002867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:49.004757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:49.073652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092381d-d4b9-4b6d-8bdc-838ecee24ebb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:49.078047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17e1cd-de65-4864-944f-eca53a35c8f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:49.078685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17e1cd-de65-4864-944f-eca53a35c8f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:49.078963 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17e1cd-de65-4864-944f-eca53a35c8f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:49.079268 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17e1cd-de65-4864-944f-eca53a35c8f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:49.094269 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17e1cd-de65-4864-944f-eca53a35c8f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:49.096366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17e1cd-de65-4864-944f-eca53a35c8f9 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:49.108142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 929/1881] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:49.450217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ff6cb-08ce-4f7f-952e-0aac71090474 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:49.451714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-734ff6cb-08ce-4f7f-952e-0aac71090474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:49.465894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ff6cb-08ce-4f7f-952e-0aac71090474 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:49.470806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:49.471440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:49.471856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:49.472007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] PATH_INFO: `/v3/projects/d38fddb6eaab4f37986f373ec3b6af44` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:49.473123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d38fddb6eaab4f37986f373ec3b6af44)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:49.475125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:49.523802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd38fddb6eaab4f37986f373ec3b6af44', 'request_id': 'req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:49.530495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd38fddb6eaab4f37986f373ec3b6af44', 'request_id': 'req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:49.536788 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd38fddb6eaab4f37986f373ec3b6af44', 'request_id': 'req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:49.550968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] The token cache is being invalidate because project d38fddb6eaab4f37986f373ec3b6af44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:49.551256 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:49.551880 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:49.553484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 953/1882] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:49 2022] DELETE /identity/v3/projects/d38fddb6eaab4f37986f373ec3b6af44 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:49.557689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17e1cd-de65-4864-944f-eca53a35c8f9 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:49.567351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17e1cd-de65-4864-944f-eca53a35c8f9 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:49.591822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:49.630567 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd330ca-582f-4116-bbe9-2e2f204a58f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:49.631083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd330ca-582f-4116-bbe9-2e2f204a58f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:49.631361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd330ca-582f-4116-bbe9-2e2f204a58f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:49.631619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd330ca-582f-4116-bbe9-2e2f204a58f5 admin admin] PATH_INFO: `/v3/users/f669fdee32704583bfb471651f74226a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:49.633533 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd330ca-582f-4116-bbe9-2e2f204a58f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f669fdee32704583bfb471651f74226a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:49.634790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd330ca-582f-4116-bbe9-2e2f204a58f5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:49.688723 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1cd330ca-582f-4116-bbe9-2e2f204a58f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f669fdee32704583bfb471651f74226a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:49.689390 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1cd330ca-582f-4116-bbe9-2e2f204a58f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f669fdee32704583bfb471651f74226a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:49.696497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1cd330ca-582f-4116-bbe9-2e2f204a58f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f669fdee32704583bfb471651f74226a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:49.705868 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1cd330ca-582f-4116-bbe9-2e2f204a58f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f669fdee32704583bfb471651f74226a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:49.718769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1cd330ca-582f-4116-bbe9-2e2f204a58f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f669fdee32704583bfb471651f74226a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:49.724740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 930/1883] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:24:49 2022] DELETE /identity/v3/users/f669fdee32704583bfb471651f74226a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:49.997691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:49.999160 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:50.013616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8c54f-f678-4684-8fe0-ac75d73fb7c7 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:50.072755 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:50.073386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:50.073665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:50.073969 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468 admin admin] PATH_INFO: `/v3/projects/0d0eaeb82af44bc0a9012a19adc59c7c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:50.075104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d0eaeb82af44bc0a9012a19adc59c7c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:50.077003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:50.088044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:50.126492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d0eaeb82af44bc0a9012a19adc59c7c', 'request_id': 'req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:50.131948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d0eaeb82af44bc0a9012a19adc59c7c', 'request_id': 'req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:50.138767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d0eaeb82af44bc0a9012a19adc59c7c', 'request_id': 'req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:50.153150 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468 admin admin] The token cache is being invalidate because project 0d0eaeb82af44bc0a9012a19adc59c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:24:50.153424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:50.154041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-de72de0c-4d38-41ac-ba52-fa2c2a1fe468 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:24:50.155613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 954/1884] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:24:49 2022] DELETE /identity/v3/projects/0d0eaeb82af44bc0a9012a19adc59c7c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:50.161728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2595654-a7c5-4e4e-9c25-55354da4e9b4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:50.162013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2595654-a7c5-4e4e-9c25-55354da4e9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:50.162316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2595654-a7c5-4e4e-9c25-55354da4e9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:50.188926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e2595654-a7c5-4e4e-9c25-55354da4e9b4 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:50.190172 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e2595654-a7c5-4e4e-9c25-55354da4e9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:50.258116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 931/1885] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:24:50.265749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe93091-4de6-428b-8e4f-e079aac68885 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:50.266235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe93091-4de6-428b-8e4f-e079aac68885 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:50.266753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe93091-4de6-428b-8e4f-e079aac68885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:50.298552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-fbe93091-4de6-428b-8e4f-e079aac68885 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:50.299759 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fbe93091-4de6-428b-8e4f-e079aac68885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:50.307870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 955/1886] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:50.312176 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2595654-a7c5-4e4e-9c25-55354da4e9b4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:50.323925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2595654-a7c5-4e4e-9c25-55354da4e9b4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:50.328611 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dbf47-3049-4aad-8934-3a85bf960879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:50.329214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dbf47-3049-4aad-8934-3a85bf960879 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:50.329520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dbf47-3049-4aad-8934-3a85bf960879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:50.329790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04dbf47-3049-4aad-8934-3a85bf960879 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:50.330472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dbf47-3049-4aad-8934-3a85bf960879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:24:50.330771 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dbf47-3049-4aad-8934-3a85bf960879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:50.332356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04dbf47-3049-4aad-8934-3a85bf960879 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:50.345458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 932/1887] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:24:50.349643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe93091-4de6-428b-8e4f-e079aac68885 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:50.360030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe93091-4de6-428b-8e4f-e079aac68885 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:50.364656 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d7bcc-0a71-428d-9f75-e4315323310d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:50.365252 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d7bcc-0a71-428d-9f75-e4315323310d admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:50.365526 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d7bcc-0a71-428d-9f75-e4315323310d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:50.365796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d7bcc-0a71-428d-9f75-e4315323310d admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:50.366466 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d7bcc-0a71-428d-9f75-e4315323310d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:50.368388 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d7bcc-0a71-428d-9f75-e4315323310d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:50.398119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 956/1888] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:24:50 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:50.401160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dbf47-3049-4aad-8934-3a85bf960879 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:50.412137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04dbf47-3049-4aad-8934-3a85bf960879 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:50.445362 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:50.485430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482eb4ba-f89a-4d43-b78f-909a9ef4544d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:50.486045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482eb4ba-f89a-4d43-b78f-909a9ef4544d admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:50.486351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482eb4ba-f89a-4d43-b78f-909a9ef4544d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:50.486618 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482eb4ba-f89a-4d43-b78f-909a9ef4544d admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:50.487252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482eb4ba-f89a-4d43-b78f-909a9ef4544d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:50.489201 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482eb4ba-f89a-4d43-b78f-909a9ef4544d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:50.513267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 933/1889] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:24:50 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:24:50.517466 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d7bcc-0a71-428d-9f75-e4315323310d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:50.528419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d7bcc-0a71-428d-9f75-e4315323310d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:50.532977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981deca-b671-412d-9bf1-d2a236051993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:50.533569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981deca-b671-412d-9bf1-d2a236051993 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:50.533846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981deca-b671-412d-9bf1-d2a236051993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:50.534155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981deca-b671-412d-9bf1-d2a236051993 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:50.534797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981deca-b671-412d-9bf1-d2a236051993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:50.536389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981deca-b671-412d-9bf1-d2a236051993 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:50.546922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 957/1890] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:24:50.550797 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482eb4ba-f89a-4d43-b78f-909a9ef4544d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:50.560872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482eb4ba-f89a-4d43-b78f-909a9ef4544d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:50.565444 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d858172-bc30-473f-badd-8823490c151f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:50.566034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d858172-bc30-473f-badd-8823490c151f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:50.566345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d858172-bc30-473f-badd-8823490c151f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:50.566644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d858172-bc30-473f-badd-8823490c151f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:50.567276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d858172-bc30-473f-badd-8823490c151f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:50.568838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d858172-bc30-473f-badd-8823490c151f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:50.579722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 934/1891] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:24:50.583651 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981deca-b671-412d-9bf1-d2a236051993 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:50.593992 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981deca-b671-412d-9bf1-d2a236051993 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:50.598646 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e0081-4a25-4ab7-982b-08089140dbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:50.599284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e0081-4a25-4ab7-982b-08089140dbab admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:50.599564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e0081-4a25-4ab7-982b-08089140dbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:50.599838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e0081-4a25-4ab7-982b-08089140dbab admin admin] PATH_INFO: `/v3/projects/7d31e5cd47024ab49127c5ee7f0bbc87/users/961ce0af5d934b45999a0630b5101fa2/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:50.614278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e0081-4a25-4ab7-982b-08089140dbab admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d31e5cd47024ab49127c5ee7f0bbc87, user_id=961ce0af5d934b45999a0630b5101fa2, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:50.616918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e0081-4a25-4ab7-982b-08089140dbab admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:50.629871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 958/1892] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:24:50 2022] PUT /identity/v3/projects/7d31e5cd47024ab49127c5ee7f0bbc87/users/961ce0af5d934b45999a0630b5101fa2/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:24:50.640461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ad84b-5295-45fe-9d2f-9b4dfb520611 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:50.640740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ad84b-5295-45fe-9d2f-9b4dfb520611 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:50.641059 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ad84b-5295-45fe-9d2f-9b4dfb520611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:50.669225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-353ad84b-5295-45fe-9d2f-9b4dfb520611 None None] MFA Rules not processed for user `961ce0af5d934b45999a0630b5101fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:24:50.670469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-353ad84b-5295-45fe-9d2f-9b4dfb520611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:50.737942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 935/1893] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:24:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:24:50.749086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e0081-4a25-4ab7-982b-08089140dbab admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:50.750800 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c62e0081-4a25-4ab7-982b-08089140dbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:50.826843 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e0081-4a25-4ab7-982b-08089140dbab admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:50.830452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a44e4a-968b-4972-bf61-9eac89486d7f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc89c1a7e684612a73ec9fef6bda780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:50.831047 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a44e4a-968b-4972-bf61-9eac89486d7f service glance] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:50.831322 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a44e4a-968b-4972-bf61-9eac89486d7f service glance] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:50.831595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a44e4a-968b-4972-bf61-9eac89486d7f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:50.840134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a44e4a-968b-4972-bf61-9eac89486d7f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:50.841577 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a44e4a-968b-4972-bf61-9eac89486d7f service glance] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:50.850273 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 959/1894] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:24:56.210704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ad84b-5295-45fe-9d2f-9b4dfb520611 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:24:56.213805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-353ad84b-5295-45fe-9d2f-9b4dfb520611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:56.287152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ad84b-5295-45fe-9d2f-9b4dfb520611 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:24:56.293311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752d398-99ea-4768-858b-9d5630a89efd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:24:56.294005 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752d398-99ea-4768-858b-9d5630a89efd service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:24:56.294379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752d398-99ea-4768-858b-9d5630a89efd service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:24:56.294715 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752d398-99ea-4768-858b-9d5630a89efd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:24:56.296651 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b752d398-99ea-4768-858b-9d5630a89efd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:24:56.369208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:24:56.385254 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752d398-99ea-4768-858b-9d5630a89efd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:24:56.387491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752d398-99ea-4768-858b-9d5630a89efd service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:24:56.402973 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 936/1895] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:24:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:00.846311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:02.514037 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d3be3-d175-4f71-969e-1fc25038fd65 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:02.514710 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d3be3-d175-4f71-969e-1fc25038fd65 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:02.514710 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d3be3-d175-4f71-969e-1fc25038fd65 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:02.516049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 960/1896] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:25:02.526983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5b810-09f5-4f9d-93e3-81f1665a63a5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:02.527441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5b810-09f5-4f9d-93e3-81f1665a63a5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:02.527665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5b810-09f5-4f9d-93e3-81f1665a63a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:02.549633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0de5b810-09f5-4f9d-93e3-81f1665a63a5 None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:02.550721 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0de5b810-09f5-4f9d-93e3-81f1665a63a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:02.624993 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 937/1897] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:25:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:02.637779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d3be3-d175-4f71-969e-1fc25038fd65 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:02.639464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-656d3be3-d175-4f71-969e-1fc25038fd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:02.660304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d3be3-d175-4f71-969e-1fc25038fd65 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:02.665690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f60bb-d223-4ba6-a5fc-c0f76971078a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:02.666315 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f60bb-d223-4ba6-a5fc-c0f76971078a service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:02.666587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f60bb-d223-4ba6-a5fc-c0f76971078a service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:02.666853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f60bb-d223-4ba6-a5fc-c0f76971078a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:02.675253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f60bb-d223-4ba6-a5fc-c0f76971078a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:02.676818 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f60bb-d223-4ba6-a5fc-c0f76971078a service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:02.687609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 961/1898] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:03.812024 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5b810-09f5-4f9d-93e3-81f1665a63a5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:03.813481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0de5b810-09f5-4f9d-93e3-81f1665a63a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:03.882372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5b810-09f5-4f9d-93e3-81f1665a63a5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:03.887189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:03.887902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:03.888182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:03.888459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] PATH_INFO: `/v3/users/bfaf4a051e0149a18851bdc11be1f122` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:03.889637 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfaf4a051e0149a18851bdc11be1f122)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:03.891747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:03.940704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfaf4a051e0149a18851bdc11be1f122'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:03.941339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfaf4a051e0149a18851bdc11be1f122'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:03.948122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfaf4a051e0149a18851bdc11be1f122'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:03.955680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfaf4a051e0149a18851bdc11be1f122'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:03.968944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfaf4a051e0149a18851bdc11be1f122'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:03.974846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 938/1899] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:03 2022] DELETE /identity/v3/users/bfaf4a051e0149a18851bdc11be1f122 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:03.979532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff11dfb-e9a6-4c8d-9502-efa1dd3bed77 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:03.980829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff11dfb-e9a6-4c8d-9502-efa1dd3bed77 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:03.980923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff11dfb-e9a6-4c8d-9502-efa1dd3bed77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:04.000593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-fff11dfb-e9a6-4c8d-9502-efa1dd3bed77 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:04.001809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fff11dfb-e9a6-4c8d-9502-efa1dd3bed77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:04.065045 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 962/1900] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:04.076154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:04.077760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:04.083202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:04.138994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8ae3b-9ea2-46a0-90f6-ca35a3c8f0f5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:04.143047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843e4fc-bbc6-4f61-8fea-c86b54c7f173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:04.143684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843e4fc-bbc6-4f61-8fea-c86b54c7f173 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:04.144032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843e4fc-bbc6-4f61-8fea-c86b54c7f173 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:04.144322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843e4fc-bbc6-4f61-8fea-c86b54c7f173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:04.155199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843e4fc-bbc6-4f61-8fea-c86b54c7f173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:04.156626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843e4fc-bbc6-4f61-8fea-c86b54c7f173 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:04.166288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 939/1901] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:04.442275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff11dfb-e9a6-4c8d-9502-efa1dd3bed77 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:04.443939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fff11dfb-e9a6-4c8d-9502-efa1dd3bed77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:04.463502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff11dfb-e9a6-4c8d-9502-efa1dd3bed77 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:04.468641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:04.469812 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:04.470261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:04.470669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] PATH_INFO: `/v3/projects/daea53c18fc1484ab977aaf49e46c204` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:04.474377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=daea53c18fc1484ab977aaf49e46c204)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:04.476498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:04.525325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daea53c18fc1484ab977aaf49e46c204', 'request_id': 'req-ddd6e492-2faf-4771-b33e-0a086f0923ea'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:04.532021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daea53c18fc1484ab977aaf49e46c204', 'request_id': 'req-ddd6e492-2faf-4771-b33e-0a086f0923ea'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:04.538739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daea53c18fc1484ab977aaf49e46c204', 'request_id': 'req-ddd6e492-2faf-4771-b33e-0a086f0923ea'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:04.552448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] The token cache is being invalidate because project daea53c18fc1484ab977aaf49e46c204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:04.552755 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:04.553365 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:04.554908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 963/1902] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:04 2022] DELETE /identity/v3/projects/daea53c18fc1484ab977aaf49e46c204 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:04.558984 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843e4fc-bbc6-4f61-8fea-c86b54c7f173 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:04.569067 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843e4fc-bbc6-4f61-8fea-c86b54c7f173 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:04.646649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a45b3-3bbb-4f59-abb4-884258e47de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:04.647411 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a45b3-3bbb-4f59-abb4-884258e47de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:04.647751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a45b3-3bbb-4f59-abb4-884258e47de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:04.648087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a45b3-3bbb-4f59-abb4-884258e47de5 admin admin] PATH_INFO: `/v3/users/547eeae85f524d8abbf5ac7893fd89a2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:04.649297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a45b3-3bbb-4f59-abb4-884258e47de5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=547eeae85f524d8abbf5ac7893fd89a2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:04.651331 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a45b3-3bbb-4f59-abb4-884258e47de5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:04.704618 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1e9a45b3-3bbb-4f59-abb4-884258e47de5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '547eeae85f524d8abbf5ac7893fd89a2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:04.705276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1e9a45b3-3bbb-4f59-abb4-884258e47de5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '547eeae85f524d8abbf5ac7893fd89a2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:04.712344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1e9a45b3-3bbb-4f59-abb4-884258e47de5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '547eeae85f524d8abbf5ac7893fd89a2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:04.721199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1e9a45b3-3bbb-4f59-abb4-884258e47de5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '547eeae85f524d8abbf5ac7893fd89a2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:04.734841 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1e9a45b3-3bbb-4f59-abb4-884258e47de5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '547eeae85f524d8abbf5ac7893fd89a2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:04.741161 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 940/1903] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:04 2022] DELETE /identity/v3/users/547eeae85f524d8abbf5ac7893fd89a2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:05.140773 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:05.142342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:05.155606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6e492-2faf-4771-b33e-0a086f0923ea admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:05.218471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc691b-e1fb-4cd3-be7e-914225057ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:05.219514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc691b-e1fb-4cd3-be7e-914225057ea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:05.220029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc691b-e1fb-4cd3-be7e-914225057ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:05.220539 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc691b-e1fb-4cd3-be7e-914225057ea5 admin admin] PATH_INFO: `/v3/projects/15da4805ff5d4f259713a8d83caa1a0f` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:05.221827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc691b-e1fb-4cd3-be7e-914225057ea5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15da4805ff5d4f259713a8d83caa1a0f)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:05.223649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc691b-e1fb-4cd3-be7e-914225057ea5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:05.235678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:05.273838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-21cc691b-e1fb-4cd3-be7e-914225057ea5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15da4805ff5d4f259713a8d83caa1a0f', 'request_id': 'req-21cc691b-e1fb-4cd3-be7e-914225057ea5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:05.279722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-21cc691b-e1fb-4cd3-be7e-914225057ea5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15da4805ff5d4f259713a8d83caa1a0f', 'request_id': 'req-21cc691b-e1fb-4cd3-be7e-914225057ea5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:05.285870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-21cc691b-e1fb-4cd3-be7e-914225057ea5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15da4805ff5d4f259713a8d83caa1a0f', 'request_id': 'req-21cc691b-e1fb-4cd3-be7e-914225057ea5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:05.299067 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-21cc691b-e1fb-4cd3-be7e-914225057ea5 admin admin] The token cache is being invalidate because project 15da4805ff5d4f259713a8d83caa1a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:05.299449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-21cc691b-e1fb-4cd3-be7e-914225057ea5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:05.300124 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-21cc691b-e1fb-4cd3-be7e-914225057ea5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:05.305521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 964/1904] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:05 2022] DELETE /identity/v3/projects/15da4805ff5d4f259713a8d83caa1a0f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:05.307193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c1423-88d7-49b9-b4dd-a7cee3992ba6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:05.308548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c1423-88d7-49b9-b4dd-a7cee3992ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:05.308548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c1423-88d7-49b9-b4dd-a7cee3992ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:05.329178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e40c1423-88d7-49b9-b4dd-a7cee3992ba6 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:05.330427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e40c1423-88d7-49b9-b4dd-a7cee3992ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:05.351578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:05.393825 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 941/1905] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:05.402817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c82498-7387-4d2a-b8b1-ba6f5d00e910 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:05.403127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c82498-7387-4d2a-b8b1-ba6f5d00e910 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:05.403408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c82498-7387-4d2a-b8b1-ba6f5d00e910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:05.422731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c4c82498-7387-4d2a-b8b1-ba6f5d00e910 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:05.423937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c4c82498-7387-4d2a-b8b1-ba6f5d00e910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:05.431689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 965/1906] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:05.436443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c1423-88d7-49b9-b4dd-a7cee3992ba6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:05.446128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c1423-88d7-49b9-b4dd-a7cee3992ba6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:05.451282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36f51a-e2a3-4c40-8cc4-6bff9ec9ad35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:05.451870 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36f51a-e2a3-4c40-8cc4-6bff9ec9ad35 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:05.452151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36f51a-e2a3-4c40-8cc4-6bff9ec9ad35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:05.452420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36f51a-e2a3-4c40-8cc4-6bff9ec9ad35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:05.453060 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36f51a-e2a3-4c40-8cc4-6bff9ec9ad35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:25:05.453359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36f51a-e2a3-4c40-8cc4-6bff9ec9ad35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:05.454971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f36f51a-e2a3-4c40-8cc4-6bff9ec9ad35 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:05.466988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 942/1907] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:25:05.471069 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c82498-7387-4d2a-b8b1-ba6f5d00e910 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:05.481537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c82498-7387-4d2a-b8b1-ba6f5d00e910 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:05.486252 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccc2ec-aa80-478e-a112-d14c0f2e93c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:05.486856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccc2ec-aa80-478e-a112-d14c0f2e93c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:05.487128 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccc2ec-aa80-478e-a112-d14c0f2e93c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:05.487395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccc2ec-aa80-478e-a112-d14c0f2e93c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:05.488034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccc2ec-aa80-478e-a112-d14c0f2e93c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:05.489934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccc2ec-aa80-478e-a112-d14c0f2e93c1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:05.523104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 966/1908] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:05 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:05.527625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36f51a-e2a3-4c40-8cc4-6bff9ec9ad35 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:05.537571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f36f51a-e2a3-4c40-8cc4-6bff9ec9ad35 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:05.595533 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc4bb5-8d8f-48de-b5eb-c8622fbe5f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:05.596169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc4bb5-8d8f-48de-b5eb-c8622fbe5f39 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:05.596448 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc4bb5-8d8f-48de-b5eb-c8622fbe5f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:05.596718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc4bb5-8d8f-48de-b5eb-c8622fbe5f39 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:05.597384 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc4bb5-8d8f-48de-b5eb-c8622fbe5f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:05.599317 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc4bb5-8d8f-48de-b5eb-c8622fbe5f39 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:05.621517 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 943/1909] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:05 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:25:05.625410 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccc2ec-aa80-478e-a112-d14c0f2e93c1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:05.636277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccc2ec-aa80-478e-a112-d14c0f2e93c1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:05.640930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0f23c-7f05-42ca-aca2-55718fca0ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:05.641525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0f23c-7f05-42ca-aca2-55718fca0ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:05.641808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0f23c-7f05-42ca-aca2-55718fca0ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:05.642101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0f23c-7f05-42ca-aca2-55718fca0ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:05.642749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0f23c-7f05-42ca-aca2-55718fca0ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:05.644334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0f23c-7f05-42ca-aca2-55718fca0ed9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:05.655499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 967/1910] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:05.659621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc4bb5-8d8f-48de-b5eb-c8622fbe5f39 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:05.670152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc4bb5-8d8f-48de-b5eb-c8622fbe5f39 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:05.674779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92417464-62fd-444e-8620-983d5cc7297e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:05.675329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92417464-62fd-444e-8620-983d5cc7297e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:05.675643 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92417464-62fd-444e-8620-983d5cc7297e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:05.675892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92417464-62fd-444e-8620-983d5cc7297e admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:05.676525 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92417464-62fd-444e-8620-983d5cc7297e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:05.678110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92417464-62fd-444e-8620-983d5cc7297e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:05.689413 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 944/1911] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:05.693563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0f23c-7f05-42ca-aca2-55718fca0ed9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:05.705678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0f23c-7f05-42ca-aca2-55718fca0ed9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:05.710195 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324037e-696e-4677-9cfc-036d6f49c545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:05.710831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324037e-696e-4677-9cfc-036d6f49c545 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:05.711104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324037e-696e-4677-9cfc-036d6f49c545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:05.711369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324037e-696e-4677-9cfc-036d6f49c545 admin admin] PATH_INFO: `/v3/projects/3634f47319c9448089b001a4764939ac/users/0e13f36f4eb348df8c50e67e7d0c2c79/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:05.726283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324037e-696e-4677-9cfc-036d6f49c545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3634f47319c9448089b001a4764939ac, user_id=0e13f36f4eb348df8c50e67e7d0c2c79, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:05.728973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324037e-696e-4677-9cfc-036d6f49c545 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:05.742110 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 968/1912] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:05 2022] PUT /identity/v3/projects/3634f47319c9448089b001a4764939ac/users/0e13f36f4eb348df8c50e67e7d0c2c79/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:05.752195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf190fff-0595-4cad-b358-4614a9888cab None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:05.752592 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf190fff-0595-4cad-b358-4614a9888cab None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:05.752872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf190fff-0595-4cad-b358-4614a9888cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:05.781218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-cf190fff-0595-4cad-b358-4614a9888cab None None] MFA Rules not processed for user `0e13f36f4eb348df8c50e67e7d0c2c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:05.782544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cf190fff-0595-4cad-b358-4614a9888cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:05.845545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 945/1913] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:05.849990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324037e-696e-4677-9cfc-036d6f49c545 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:05.909628 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324037e-696e-4677-9cfc-036d6f49c545 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:05.914043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2ba5e-85e5-45e3-8209-c08d56c97905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:05.914659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2ba5e-85e5-45e3-8209-c08d56c97905 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:05.914929 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2ba5e-85e5-45e3-8209-c08d56c97905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:05.915193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2ba5e-85e5-45e3-8209-c08d56c97905 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:05.915808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2ba5e-85e5-45e3-8209-c08d56c97905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:05.917736 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2ba5e-85e5-45e3-8209-c08d56c97905 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:05.943477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 969/1914] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:05 2022] POST /identity/v3/projects => generated 369 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:05.947529 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf190fff-0595-4cad-b358-4614a9888cab None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:05.958702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf190fff-0595-4cad-b358-4614a9888cab None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:05.974643 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:06.016939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382508cb-972b-425a-848e-d910de3173aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:06.017544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382508cb-972b-425a-848e-d910de3173aa admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:06.017822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382508cb-972b-425a-848e-d910de3173aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:06.018162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382508cb-972b-425a-848e-d910de3173aa admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:06.018751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382508cb-972b-425a-848e-d910de3173aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:06.020642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382508cb-972b-425a-848e-d910de3173aa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:06.041095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 946/1915] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:05 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:25:06.044976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2ba5e-85e5-45e3-8209-c08d56c97905 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:06.055144 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2ba5e-85e5-45e3-8209-c08d56c97905 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:06.061321 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4240fb9-9e4b-4e17-b873-faadd78a9219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:06.061911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4240fb9-9e4b-4e17-b873-faadd78a9219 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:06.062228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4240fb9-9e4b-4e17-b873-faadd78a9219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:06.062499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4240fb9-9e4b-4e17-b873-faadd78a9219 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:06.063583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4240fb9-9e4b-4e17-b873-faadd78a9219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:06.064712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4240fb9-9e4b-4e17-b873-faadd78a9219 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:06.077433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 970/1916] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:06.081631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382508cb-972b-425a-848e-d910de3173aa admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:06.092791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382508cb-972b-425a-848e-d910de3173aa admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:06.097670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a3f0d-9ebd-472b-abcb-b0ff03ad1120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:06.098472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a3f0d-9ebd-472b-abcb-b0ff03ad1120 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:06.098753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a3f0d-9ebd-472b-abcb-b0ff03ad1120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:06.099027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a3f0d-9ebd-472b-abcb-b0ff03ad1120 admin admin] PATH_INFO: `/v3/projects/7003362ce1de4bc79ef37776ec8cd1d3/users/2f9cb334d7634d27b04f5af7f2b72c3a/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:06.115354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a3f0d-9ebd-472b-abcb-b0ff03ad1120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7003362ce1de4bc79ef37776ec8cd1d3, user_id=2f9cb334d7634d27b04f5af7f2b72c3a, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:06.118400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a3f0d-9ebd-472b-abcb-b0ff03ad1120 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:06.133470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 947/1917] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:06 2022] PUT /identity/v3/projects/7003362ce1de4bc79ef37776ec8cd1d3/users/2f9cb334d7634d27b04f5af7f2b72c3a/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:06.138181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4240fb9-9e4b-4e17-b873-faadd78a9219 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:06.151897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4240fb9-9e4b-4e17-b873-faadd78a9219 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:06.163255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:06.221742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab1ea2-46bb-49bd-9ef9-d6219d307f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:06.222392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab1ea2-46bb-49bd-9ef9-d6219d307f92 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:06.222673 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab1ea2-46bb-49bd-9ef9-d6219d307f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:06.222946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab1ea2-46bb-49bd-9ef9-d6219d307f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:06.223593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab1ea2-46bb-49bd-9ef9-d6219d307f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:06.225245 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab1ea2-46bb-49bd-9ef9-d6219d307f92 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:06.235716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 971/1918] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:06.239858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a3f0d-9ebd-472b-abcb-b0ff03ad1120 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:06.251355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a3f0d-9ebd-472b-abcb-b0ff03ad1120 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:06.256469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d80d1-8542-48f0-bbc0-ea86ed1e06c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:06.257205 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d80d1-8542-48f0-bbc0-ea86ed1e06c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:06.257481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d80d1-8542-48f0-bbc0-ea86ed1e06c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:06.257754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d80d1-8542-48f0-bbc0-ea86ed1e06c0 admin admin] PATH_INFO: `/v3/projects/7003362ce1de4bc79ef37776ec8cd1d3/users/2f9cb334d7634d27b04f5af7f2b72c3a/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:06.259743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d80d1-8542-48f0-bbc0-ea86ed1e06c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7003362ce1de4bc79ef37776ec8cd1d3, user_id=2f9cb334d7634d27b04f5af7f2b72c3a, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:06.263096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d80d1-8542-48f0-bbc0-ea86ed1e06c0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:06.273961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 948/1919] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:06 2022] PUT /identity/v3/projects/7003362ce1de4bc79ef37776ec8cd1d3/users/2f9cb334d7634d27b04f5af7f2b72c3a/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:06.284395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd1354-0563-4284-a3e5-8204752f89ed None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:06.284682 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd1354-0563-4284-a3e5-8204752f89ed None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:06.284957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd1354-0563-4284-a3e5-8204752f89ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:06.324355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-62fd1354-0563-4284-a3e5-8204752f89ed None None] MFA Rules not processed for user `2f9cb334d7634d27b04f5af7f2b72c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:06.325573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-62fd1354-0563-4284-a3e5-8204752f89ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:06.393335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 972/1920] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:06.404693 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d80d1-8542-48f0-bbc0-ea86ed1e06c0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:06.406581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-036d80d1-8542-48f0-bbc0-ea86ed1e06c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:06.464593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d80d1-8542-48f0-bbc0-ea86ed1e06c0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:06.468211 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b6e58-678b-4c67-8312-6c752f8875cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:06.468809 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b6e58-678b-4c67-8312-6c752f8875cd service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:06.469088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b6e58-678b-4c67-8312-6c752f8875cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:06.469354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b6e58-678b-4c67-8312-6c752f8875cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:06.478049 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b6e58-678b-4c67-8312-6c752f8875cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:06.479975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b6e58-678b-4c67-8312-6c752f8875cd service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:06.489514 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 949/1921] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:06.536889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd1354-0563-4284-a3e5-8204752f89ed None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:06.547282 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd1354-0563-4284-a3e5-8204752f89ed None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:06.550783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88fa69-d9f8-4a01-be66-aac6def26a28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:06.551398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88fa69-d9f8-4a01-be66-aac6def26a28 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:06.551670 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88fa69-d9f8-4a01-be66-aac6def26a28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:06.551935 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88fa69-d9f8-4a01-be66-aac6def26a28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:06.608233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88fa69-d9f8-4a01-be66-aac6def26a28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:06.609650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88fa69-d9f8-4a01-be66-aac6def26a28 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:06.618370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 973/1922] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:10.111741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b6e58-678b-4c67-8312-6c752f8875cd service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:10.186991 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b6e58-678b-4c67-8312-6c752f8875cd service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:10.192889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:10.193846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:10.194169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:10.194445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] PATH_INFO: `/v3/users/0e13f36f4eb348df8c50e67e7d0c2c79` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:10.195662 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e13f36f4eb348df8c50e67e7d0c2c79)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:10.197602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:10.216685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:10.260409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e13f36f4eb348df8c50e67e7d0c2c79'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:10.261080 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e13f36f4eb348df8c50e67e7d0c2c79'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:10.268228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e13f36f4eb348df8c50e67e7d0c2c79'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:10.277095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e13f36f4eb348df8c50e67e7d0c2c79'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:10.290361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e13f36f4eb348df8c50e67e7d0c2c79'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:10.297803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 950/1923] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:10 2022] DELETE /identity/v3/users/0e13f36f4eb348df8c50e67e7d0c2c79 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:10.303420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525835e3-482a-4a95-ba5f-c657b27b18a6 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:10.304423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525835e3-482a-4a95-ba5f-c657b27b18a6 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:10.304931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525835e3-482a-4a95-ba5f-c657b27b18a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:10.332613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-525835e3-482a-4a95-ba5f-c657b27b18a6 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:10.333919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-525835e3-482a-4a95-ba5f-c657b27b18a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:10.364377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:10.405031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 974/1924] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:10.427041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:10.427041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:10.498796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b47abf-05a1-4d82-8df0-4206161d72fe admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:10.502470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1f855-2a13-40cf-b774-252bdc01a533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:10.503104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1f855-2a13-40cf-b774-252bdc01a533 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:10.503615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1f855-2a13-40cf-b774-252bdc01a533 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:10.504062 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1f855-2a13-40cf-b774-252bdc01a533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:10.514392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1f855-2a13-40cf-b774-252bdc01a533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:10.516018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1f855-2a13-40cf-b774-252bdc01a533 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:10.527066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 951/1925] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:10.803304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525835e3-482a-4a95-ba5f-c657b27b18a6 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:10.804890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-525835e3-482a-4a95-ba5f-c657b27b18a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:10.825700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525835e3-482a-4a95-ba5f-c657b27b18a6 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:10.830841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:10.831485 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:10.831760 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:10.832065 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] PATH_INFO: `/v3/projects/3634f47319c9448089b001a4764939ac` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:10.833290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=3634f47319c9448089b001a4764939ac)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:10.835472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:10.913129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3634f47319c9448089b001a4764939ac', 'request_id': 'req-908d1e72-dde1-4f43-bbf9-ce176db879ab'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:10.919902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3634f47319c9448089b001a4764939ac', 'request_id': 'req-908d1e72-dde1-4f43-bbf9-ce176db879ab'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:10.927327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3634f47319c9448089b001a4764939ac', 'request_id': 'req-908d1e72-dde1-4f43-bbf9-ce176db879ab'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:10.942347 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] The token cache is being invalidate because project 3634f47319c9448089b001a4764939ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:10.942630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:10.943284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:10.944764 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 975/1926] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:10 2022] DELETE /identity/v3/projects/3634f47319c9448089b001a4764939ac => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:10.952272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1f855-2a13-40cf-b774-252bdc01a533 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:10.962849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1f855-2a13-40cf-b774-252bdc01a533 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:11.014494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:11.015933 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:11.064955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e7122-3ca4-4757-a534-a338dd11e904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:11.065728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e7122-3ca4-4757-a534-a338dd11e904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:11.066440 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e7122-3ca4-4757-a534-a338dd11e904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:11.066720 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e7122-3ca4-4757-a534-a338dd11e904 admin admin] PATH_INFO: `/v3/users/2f9cb334d7634d27b04f5af7f2b72c3a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:11.067982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e7122-3ca4-4757-a534-a338dd11e904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9cb334d7634d27b04f5af7f2b72c3a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:11.070267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e7122-3ca4-4757-a534-a338dd11e904 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:11.127538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:11.138960 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a17e7122-3ca4-4757-a534-a338dd11e904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9cb334d7634d27b04f5af7f2b72c3a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.139884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a17e7122-3ca4-4757-a534-a338dd11e904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9cb334d7634d27b04f5af7f2b72c3a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.144838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d1e72-dde1-4f43-bbf9-ce176db879ab admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:11.151464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a17e7122-3ca4-4757-a534-a338dd11e904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9cb334d7634d27b04f5af7f2b72c3a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.158736 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:11.159472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:11.159749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:11.160101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] PATH_INFO: `/v3/users/2c5f043c4e464ce4ac6e570ccbbadc34` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:11.162585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c5f043c4e464ce4ac6e570ccbbadc34)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:11.163302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a17e7122-3ca4-4757-a534-a338dd11e904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9cb334d7634d27b04f5af7f2b72c3a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.167155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:11.180644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a17e7122-3ca4-4757-a534-a338dd11e904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9cb334d7634d27b04f5af7f2b72c3a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.199792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 952/1927] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:10 2022] DELETE /identity/v3/users/2f9cb334d7634d27b04f5af7f2b72c3a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:11.261755 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c5f043c4e464ce4ac6e570ccbbadc34'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.273907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c5f043c4e464ce4ac6e570ccbbadc34'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.275098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c5f043c4e464ce4ac6e570ccbbadc34'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.277544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:11.289614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c5f043c4e464ce4ac6e570ccbbadc34'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.329108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c5f043c4e464ce4ac6e570ccbbadc34'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.338966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 976/1928] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:11 2022] DELETE /identity/v3/users/2c5f043c4e464ce4ac6e570ccbbadc34 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:11.343699 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1943b-e49c-46df-8175-db056c9eb6a1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:11.343981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1943b-e49c-46df-8175-db056c9eb6a1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:11.344265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b1943b-e49c-46df-8175-db056c9eb6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:11.365406 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-25b1943b-e49c-46df-8175-db056c9eb6a1 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:11.366707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-25b1943b-e49c-46df-8175-db056c9eb6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:11.472165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 953/1929] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:11.482976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:11.484404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:11.553570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ddb01b-f435-4239-9a7f-340ded5e1ec5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:11.557185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b53b4-688f-4707-9e7b-051bfffd2311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:11.557784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b53b4-688f-4707-9e7b-051bfffd2311 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:11.558058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b53b4-688f-4707-9e7b-051bfffd2311 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:11.558364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b53b4-688f-4707-9e7b-051bfffd2311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:11.569034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b53b4-688f-4707-9e7b-051bfffd2311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:11.570660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b53b4-688f-4707-9e7b-051bfffd2311 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:11.580370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 977/1930] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:11.702233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1943b-e49c-46df-8175-db056c9eb6a1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:11.703896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-25b1943b-e49c-46df-8175-db056c9eb6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:11.717803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b1943b-e49c-46df-8175-db056c9eb6a1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:11.727334 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:11.728155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:11.728482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:11.728777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] PATH_INFO: `/v3/projects/7003362ce1de4bc79ef37776ec8cd1d3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:11.730011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7003362ce1de4bc79ef37776ec8cd1d3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:11.732840 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:11.810347 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7003362ce1de4bc79ef37776ec8cd1d3', 'request_id': 'req-70298f27-c640-4754-bc83-15b7579a84b9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.818063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7003362ce1de4bc79ef37776ec8cd1d3', 'request_id': 'req-70298f27-c640-4754-bc83-15b7579a84b9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.824797 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7003362ce1de4bc79ef37776ec8cd1d3', 'request_id': 'req-70298f27-c640-4754-bc83-15b7579a84b9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.846159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] The token cache is being invalidate because project 7003362ce1de4bc79ef37776ec8cd1d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:11.846593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.847339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:11.850104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 954/1931] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:11 2022] DELETE /identity/v3/projects/7003362ce1de4bc79ef37776ec8cd1d3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:11.855569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbaf75-15a9-4f1c-a3bf-b28215acf8c9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:11.856434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbaf75-15a9-4f1c-a3bf-b28215acf8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:11.856735 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbaf75-15a9-4f1c-a3bf-b28215acf8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:11.884319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-46fbaf75-15a9-4f1c-a3bf-b28215acf8c9 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:11.885541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-46fbaf75-15a9-4f1c-a3bf-b28215acf8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:11.943661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:11.945375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:11.964111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 978/1932] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:11.973775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52613460-c9b8-4514-b4de-aaf90844265d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:11.974056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52613460-c9b8-4514-b4de-aaf90844265d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:11.974367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52613460-c9b8-4514-b4de-aaf90844265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.003182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-52613460-c9b8-4514-b4de-aaf90844265d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:12.004417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-52613460-c9b8-4514-b4de-aaf90844265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:12.018633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 979/1933] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:12.023231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52613460-c9b8-4514-b4de-aaf90844265d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.026038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:12.033941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52613460-c9b8-4514-b4de-aaf90844265d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.038950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85ade3-8d4c-4150-9476-f223cced512d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.039580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85ade3-8d4c-4150-9476-f223cced512d admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.039858 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85ade3-8d4c-4150-9476-f223cced512d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.040128 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85ade3-8d4c-4150-9476-f223cced512d admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.040801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85ade3-8d4c-4150-9476-f223cced512d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:25:12.041101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85ade3-8d4c-4150-9476-f223cced512d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.042861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85ade3-8d4c-4150-9476-f223cced512d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.053986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70298f27-c640-4754-bc83-15b7579a84b9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.056640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 980/1934] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:25:12.061525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85ade3-8d4c-4150-9476-f223cced512d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.062206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.062868 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.063508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.063742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17 admin admin] PATH_INFO: `/v3/projects/fd4a3ae5c3ee44c9829322a938a836e4` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.064936 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd4a3ae5c3ee44c9829322a938a836e4)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.070371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.076520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85ade3-8d4c-4150-9476-f223cced512d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.081554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5051f8-2982-4816-8ca2-42fe2c5d8dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.082379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5051f8-2982-4816-8ca2-42fe2c5d8dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.082501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5051f8-2982-4816-8ca2-42fe2c5d8dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.082745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5051f8-2982-4816-8ca2-42fe2c5d8dc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.083404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5051f8-2982-4816-8ca2-42fe2c5d8dc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.085330 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5051f8-2982-4816-8ca2-42fe2c5d8dc9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.116409 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 981/1935] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:12 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:25:12.121156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5051f8-2982-4816-8ca2-42fe2c5d8dc9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.138359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd4a3ae5c3ee44c9829322a938a836e4', 'request_id': 'req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:12.143944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd4a3ae5c3ee44c9829322a938a836e4', 'request_id': 'req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:12.150271 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd4a3ae5c3ee44c9829322a938a836e4', 'request_id': 'req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:12.163478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17 admin admin] The token cache is being invalidate because project fd4a3ae5c3ee44c9829322a938a836e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:12.163819 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:12.166996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4b8fd996-b21f-458c-a06f-50e8a1e1ec17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:12.168857 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 955/1936] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:11 2022] DELETE /identity/v3/projects/fd4a3ae5c3ee44c9829322a938a836e4 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:25:12.169800 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:12.174173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e086eed-f251-41a1-9e02-5aef7ee8fecc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.174508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e086eed-f251-41a1-9e02-5aef7ee8fecc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.174836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e086eed-f251-41a1-9e02-5aef7ee8fecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.195496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0e086eed-f251-41a1-9e02-5aef7ee8fecc None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:12.196213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5051f8-2982-4816-8ca2-42fe2c5d8dc9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.196776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0e086eed-f251-41a1-9e02-5aef7ee8fecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:12.201420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709b674-aa89-4e13-90d9-4fb8b1fece03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.202022 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709b674-aa89-4e13-90d9-4fb8b1fece03 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.202357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709b674-aa89-4e13-90d9-4fb8b1fece03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.202630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709b674-aa89-4e13-90d9-4fb8b1fece03 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.203261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709b674-aa89-4e13-90d9-4fb8b1fece03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.205159 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709b674-aa89-4e13-90d9-4fb8b1fece03 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.205310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 956/1937] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:12.215518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e773a-ae89-4d7f-9a93-a955e7043661 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.215518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e773a-ae89-4d7f-9a93-a955e7043661 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.215518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396e773a-ae89-4d7f-9a93-a955e7043661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.230468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 982/1938] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:12 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:12.232022 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709b674-aa89-4e13-90d9-4fb8b1fece03 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.233630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b709b674-aa89-4e13-90d9-4fb8b1fece03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:12.234545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-396e773a-ae89-4d7f-9a93-a955e7043661 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:12.235747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-396e773a-ae89-4d7f-9a93-a955e7043661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:12.243618 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 957/1939] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:12.245567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709b674-aa89-4e13-90d9-4fb8b1fece03 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.247691 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e773a-ae89-4d7f-9a93-a955e7043661 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.250324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a09a4-6133-48bb-b375-2b7a56108c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.250920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a09a4-6133-48bb-b375-2b7a56108c1b admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.251220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a09a4-6133-48bb-b375-2b7a56108c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.251625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a09a4-6133-48bb-b375-2b7a56108c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.252257 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a09a4-6133-48bb-b375-2b7a56108c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.254146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a09a4-6133-48bb-b375-2b7a56108c1b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.261339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396e773a-ae89-4d7f-9a93-a955e7043661 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.265713 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 983/1940] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:25:12.266825 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec3c38-236c-4cdb-a6ed-2f0236e87782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.267429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec3c38-236c-4cdb-a6ed-2f0236e87782 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.267706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec3c38-236c-4cdb-a6ed-2f0236e87782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.267976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec3c38-236c-4cdb-a6ed-2f0236e87782 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.268680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec3c38-236c-4cdb-a6ed-2f0236e87782 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:25:12.268922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec3c38-236c-4cdb-a6ed-2f0236e87782 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.269806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a09a4-6133-48bb-b375-2b7a56108c1b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.270554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec3c38-236c-4cdb-a6ed-2f0236e87782 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.280346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a09a4-6133-48bb-b375-2b7a56108c1b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.284032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 958/1941] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:25:12.285001 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe980d9-842a-44f1-9c9a-1ef9fe9a7c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.285591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe980d9-842a-44f1-9c9a-1ef9fe9a7c20 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.285872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe980d9-842a-44f1-9c9a-1ef9fe9a7c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.286178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe980d9-842a-44f1-9c9a-1ef9fe9a7c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.286822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe980d9-842a-44f1-9c9a-1ef9fe9a7c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.288199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec3c38-236c-4cdb-a6ed-2f0236e87782 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.288418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe980d9-842a-44f1-9c9a-1ef9fe9a7c20 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.302854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 984/1942] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:25:12.303063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec3c38-236c-4cdb-a6ed-2f0236e87782 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.305562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6992f-3be5-4792-951c-6997b7fc6332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.306226 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6992f-3be5-4792-951c-6997b7fc6332 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.306504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6992f-3be5-4792-951c-6997b7fc6332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.306771 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6992f-3be5-4792-951c-6997b7fc6332 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.307414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6992f-3be5-4792-951c-6997b7fc6332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.309508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6992f-3be5-4792-951c-6997b7fc6332 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.310012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe980d9-842a-44f1-9c9a-1ef9fe9a7c20 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.323949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe980d9-842a-44f1-9c9a-1ef9fe9a7c20 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.330313 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19fc88-aa47-4988-b251-c4cb701cd38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.330981 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19fc88-aa47-4988-b251-c4cb701cd38e admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.331264 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19fc88-aa47-4988-b251-c4cb701cd38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.331679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19fc88-aa47-4988-b251-c4cb701cd38e admin admin] PATH_INFO: `/v3/projects/78c80cdd46024f45b9d81e8ab4000a68/users/23c7f4d5aeb14dedad7f04c049dd2bff/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.346253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 959/1943] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:12 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:25:12.347104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6992f-3be5-4792-951c-6997b7fc6332 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.348648 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-78a6992f-3be5-4792-951c-6997b7fc6332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:12.351519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19fc88-aa47-4988-b251-c4cb701cd38e admin admin] RBAC: Authorizing `identity:create_grant(project_id=78c80cdd46024f45b9d81e8ab4000a68, user_id=23c7f4d5aeb14dedad7f04c049dd2bff, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.354238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19fc88-aa47-4988-b251-c4cb701cd38e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.374515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 985/1944] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:12 2022] PUT /identity/v3/projects/78c80cdd46024f45b9d81e8ab4000a68/users/23c7f4d5aeb14dedad7f04c049dd2bff/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:12.375817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19fc88-aa47-4988-b251-c4cb701cd38e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.422318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6992f-3be5-4792-951c-6997b7fc6332 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.451182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:12.456894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19fc88-aa47-4988-b251-c4cb701cd38e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.462699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a14e0-6535-4206-97af-77b620afb74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.463546 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a14e0-6535-4206-97af-77b620afb74d admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.463830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a14e0-6535-4206-97af-77b620afb74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.464098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a14e0-6535-4206-97af-77b620afb74d admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.464736 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a14e0-6535-4206-97af-77b620afb74d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.467294 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a14e0-6535-4206-97af-77b620afb74d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.497688 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 986/1945] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:12 2022] POST /identity/v3/users => generated 303 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:12.499469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112937f4-f5fc-4a8f-abdb-ce78699c75a2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.499750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112937f4-f5fc-4a8f-abdb-ce78699c75a2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.500019 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112937f4-f5fc-4a8f-abdb-ce78699c75a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.508356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.508994 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.509271 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.509540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] PATH_INFO: `/v3/users/dd30b028afe14975b8b9d372f743023f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.510690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd30b028afe14975b8b9d372f743023f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.513715 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.535154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-112937f4-f5fc-4a8f-abdb-ce78699c75a2 None None] MFA Rules not processed for user `23c7f4d5aeb14dedad7f04c049dd2bff`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:12.536463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-112937f4-f5fc-4a8f-abdb-ce78699c75a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:12.573328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd30b028afe14975b8b9d372f743023f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:12.574125 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd30b028afe14975b8b9d372f743023f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:12.583266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd30b028afe14975b8b9d372f743023f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:12.596210 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd30b028afe14975b8b9d372f743023f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:12.612411 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd30b028afe14975b8b9d372f743023f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:12.629465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 960/1946] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:12 2022] DELETE /identity/v3/users/dd30b028afe14975b8b9d372f743023f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:12.629465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.630925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:12.643365 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:12.650382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 987/1947] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:12.655167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fe7cd-c7f1-4784-acea-a4004db14480 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.655568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fe7cd-c7f1-4784-acea-a4004db14480 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.655842 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fe7cd-c7f1-4784-acea-a4004db14480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.674201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-169fe7cd-c7f1-4784-acea-a4004db14480 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:12.675377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-169fe7cd-c7f1-4784-acea-a4004db14480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:12.695899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecd903-1117-4921-84da-3d3c7db6bb2b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.701439 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16ee6e-127c-479e-a486-6e17e26c8f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.702096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16ee6e-127c-479e-a486-6e17e26c8f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.702402 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16ee6e-127c-479e-a486-6e17e26c8f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.702723 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16ee6e-127c-479e-a486-6e17e26c8f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.703366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16ee6e-127c-479e-a486-6e17e26c8f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.705001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16ee6e-127c-479e-a486-6e17e26c8f83 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.715987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 961/1948] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:12.716814 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16ee6e-127c-479e-a486-6e17e26c8f83 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.720434 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3f16ee6e-127c-479e-a486-6e17e26c8f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:12.733053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16ee6e-127c-479e-a486-6e17e26c8f83 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.736784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 988/1949] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:12.737625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fe7cd-c7f1-4784-acea-a4004db14480 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.739258 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abba4b-cb7c-4b61-9f3d-00b6ed7cb7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.745861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abba4b-cb7c-4b61-9f3d-00b6ed7cb7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.746281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abba4b-cb7c-4b61-9f3d-00b6ed7cb7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.747052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abba4b-cb7c-4b61-9f3d-00b6ed7cb7e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.747751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fe7cd-c7f1-4784-acea-a4004db14480 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.747884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abba4b-cb7c-4b61-9f3d-00b6ed7cb7e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.749794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4abba4b-cb7c-4b61-9f3d-00b6ed7cb7e1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.752551 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47486ec3-81aa-4442-9820-38edd1a9d992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.753141 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47486ec3-81aa-4442-9820-38edd1a9d992 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.753423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47486ec3-81aa-4442-9820-38edd1a9d992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.753692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47486ec3-81aa-4442-9820-38edd1a9d992 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.754400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47486ec3-81aa-4442-9820-38edd1a9d992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.756003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47486ec3-81aa-4442-9820-38edd1a9d992 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.767173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 989/1950] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:12.767921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47486ec3-81aa-4442-9820-38edd1a9d992 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.769812 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-47486ec3-81aa-4442-9820-38edd1a9d992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:12.785415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 962/1951] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:12 2022] POST /identity/v3/projects => generated 387 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:12.786821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abba4b-cb7c-4b61-9f3d-00b6ed7cb7e1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.816203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:12.884680 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47486ec3-81aa-4442-9820-38edd1a9d992 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.907490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4abba4b-cb7c-4b61-9f3d-00b6ed7cb7e1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:12.920133 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b27ab-5a31-4448-a301-e7f78dc839df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.921509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b27ab-5a31-4448-a301-e7f78dc839df admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.921792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b27ab-5a31-4448-a301-e7f78dc839df admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.922096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b27ab-5a31-4448-a301-e7f78dc839df admin admin] PATH_INFO: `/v3/projects/3ddac4f035c44df5a409c0ed272ee4cf/users/9c588adcfeaf4596a69f55355d2dce08/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.959281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b27ab-5a31-4448-a301-e7f78dc839df admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ddac4f035c44df5a409c0ed272ee4cf, user_id=9c588adcfeaf4596a69f55355d2dce08, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.962159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7b27ab-5a31-4448-a301-e7f78dc839df admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:12.966806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8339bc7-ff41-4622-85b6-b382d7e07058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:12.967436 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339bc7-ff41-4622-85b6-b382d7e07058 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:12.967979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339bc7-ff41-4622-85b6-b382d7e07058 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:12.968284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339bc7-ff41-4622-85b6-b382d7e07058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:12.987589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 963/1952] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:12 2022] PUT /identity/v3/projects/3ddac4f035c44df5a409c0ed272ee4cf/users/9c588adcfeaf4596a69f55355d2dce08/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:12.989230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b27ab-5a31-4448-a301-e7f78dc839df admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:12.994332 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8339bc7-ff41-4622-85b6-b382d7e07058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:12.996506 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8339bc7-ff41-4622-85b6-b382d7e07058 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:13.009310 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 990/1953] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:13.011778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd837b-d320-4c27-8a95-ceb644000229 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:13.012641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd837b-d320-4c27-8a95-ceb644000229 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:13.012641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd837b-d320-4c27-8a95-ceb644000229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:13.045460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6dfd837b-d320-4c27-8a95-ceb644000229 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:13.047080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6dfd837b-d320-4c27-8a95-ceb644000229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:13.168265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:13.199055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 991/1954] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:13.219689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b27ab-5a31-4448-a301-e7f78dc839df admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:13.228013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd837b-d320-4c27-8a95-ceb644000229 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:13.229527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6dfd837b-d320-4c27-8a95-ceb644000229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:13.240837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2216adf-c97e-490e-87d4-cce1bbec7654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:13.241469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2216adf-c97e-490e-87d4-cce1bbec7654 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:13.241753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2216adf-c97e-490e-87d4-cce1bbec7654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:13.242024 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2216adf-c97e-490e-87d4-cce1bbec7654 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:13.242705 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2216adf-c97e-490e-87d4-cce1bbec7654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:13.244783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2216adf-c97e-490e-87d4-cce1bbec7654 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:13.294830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 964/1955] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:12 2022] POST /identity/v3/users => generated 294 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:13.301884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2216adf-c97e-490e-87d4-cce1bbec7654 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:13.322739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2216adf-c97e-490e-87d4-cce1bbec7654 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:13.341649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304840b5-7d22-41b5-9b9b-4cffc68650fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:13.342312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304840b5-7d22-41b5-9b9b-4cffc68650fb admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:13.342590 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304840b5-7d22-41b5-9b9b-4cffc68650fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:13.342863 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304840b5-7d22-41b5-9b9b-4cffc68650fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:13.343518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304840b5-7d22-41b5-9b9b-4cffc68650fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:13.345125 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304840b5-7d22-41b5-9b9b-4cffc68650fb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:13.359100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd837b-d320-4c27-8a95-ceb644000229 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:13.364316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05138920-510d-47cb-826c-f7f762269c8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:13.365341 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 965/1956] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:13.368815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05138920-510d-47cb-826c-f7f762269c8e service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:13.369206 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05138920-510d-47cb-826c-f7f762269c8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:13.369320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304840b5-7d22-41b5-9b9b-4cffc68650fb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:13.369603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05138920-510d-47cb-826c-f7f762269c8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:13.382483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05138920-510d-47cb-826c-f7f762269c8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:13.384355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05138920-510d-47cb-826c-f7f762269c8e service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:13.397848 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304840b5-7d22-41b5-9b9b-4cffc68650fb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:13.401974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 992/1957] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:25:13.410287 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe5788-701f-41f2-ae13-0ea9531f0f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:13.411007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe5788-701f-41f2-ae13-0ea9531f0f58 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:13.411311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe5788-701f-41f2-ae13-0ea9531f0f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:13.411690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe5788-701f-41f2-ae13-0ea9531f0f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:13.412442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe5788-701f-41f2-ae13-0ea9531f0f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:13.414168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe5788-701f-41f2-ae13-0ea9531f0f58 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:13.436654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 966/1958] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:25:13.443992 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05138920-510d-47cb-826c-f7f762269c8e service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:13.455884 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05138920-510d-47cb-826c-f7f762269c8e service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:13.462956 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650a053-7726-413c-a140-6f5e668b756f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:13.463832 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650a053-7726-413c-a140-6f5e668b756f admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:13.464234 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650a053-7726-413c-a140-6f5e668b756f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:13.464507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650a053-7726-413c-a140-6f5e668b756f admin admin] PATH_INFO: `/v3/projects/b858a38fa5594ee9a297f1f9118b8944/users/1741349b41f248d6837fc7615f0f3df5/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:13.488797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650a053-7726-413c-a140-6f5e668b756f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b858a38fa5594ee9a297f1f9118b8944, user_id=1741349b41f248d6837fc7615f0f3df5, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:13.491689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650a053-7726-413c-a140-6f5e668b756f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:13.508667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 993/1959] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:13 2022] PUT /identity/v3/projects/b858a38fa5594ee9a297f1f9118b8944/users/1741349b41f248d6837fc7615f0f3df5/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:13.520640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b08166-aaeb-4bf8-8d7f-7ba57b007a7b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:13.520950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b08166-aaeb-4bf8-8d7f-7ba57b007a7b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:13.521294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b08166-aaeb-4bf8-8d7f-7ba57b007a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:13.532834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650a053-7726-413c-a140-6f5e668b756f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:13.535985 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7650a053-7726-413c-a140-6f5e668b756f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:13.559877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-52b08166-aaeb-4bf8-8d7f-7ba57b007a7b None None] MFA Rules not processed for user `1741349b41f248d6837fc7615f0f3df5`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:13.561350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-52b08166-aaeb-4bf8-8d7f-7ba57b007a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:13.638987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650a053-7726-413c-a140-6f5e668b756f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:13.650704 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:13.651919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:13.653580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:13.653997 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] PATH_INFO: `/v3/projects/9407976c42dc41d7ad66e99b54657dbd` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:13.655610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 967/1960] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:13.659111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9407976c42dc41d7ad66e99b54657dbd)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:13.661481 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:13.664591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b08166-aaeb-4bf8-8d7f-7ba57b007a7b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:13.675610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:13.695339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b08166-aaeb-4bf8-8d7f-7ba57b007a7b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:13.705000 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27ba82-dd80-4239-a11a-84d24e6e1e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:13.705690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27ba82-dd80-4239-a11a-84d24e6e1e04 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:13.706033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27ba82-dd80-4239-a11a-84d24e6e1e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:13.706385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27ba82-dd80-4239-a11a-84d24e6e1e04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:13.707082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27ba82-dd80-4239-a11a-84d24e6e1e04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:13.709063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27ba82-dd80-4239-a11a-84d24e6e1e04 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:13.747348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9407976c42dc41d7ad66e99b54657dbd', 'request_id': 'req-f706d33e-23c7-448a-a7f7-edd844af97a3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:13.758857 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9407976c42dc41d7ad66e99b54657dbd', 'request_id': 'req-f706d33e-23c7-448a-a7f7-edd844af97a3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:13.766136 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9407976c42dc41d7ad66e99b54657dbd', 'request_id': 'req-f706d33e-23c7-448a-a7f7-edd844af97a3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:13.766393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 968/1961] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:13 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:13.767398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27ba82-dd80-4239-a11a-84d24e6e1e04 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:13.770188 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:13.771320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8d27ba82-dd80-4239-a11a-84d24e6e1e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:13.782189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] The token cache is being invalidate because project 9407976c42dc41d7ad66e99b54657dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:13.782901 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:13.784822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:13.790323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 994/1962] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:13 2022] DELETE /identity/v3/projects/9407976c42dc41d7ad66e99b54657dbd => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:13.791498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:13.794026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:13.903318 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706d33e-23c7-448a-a7f7-edd844af97a3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:13.910372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a0b92-9514-468c-ab63-587268548b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:13.911258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a0b92-9514-468c-ab63-587268548b79 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:13.911541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a0b92-9514-468c-ab63-587268548b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:13.911815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129a0b92-9514-468c-ab63-587268548b79 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:13.912655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a0b92-9514-468c-ab63-587268548b79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:13.915090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129a0b92-9514-468c-ab63-587268548b79 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:13.915624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27ba82-dd80-4239-a11a-84d24e6e1e04 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:13.931363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:13.932140 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:13.932420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:13.932803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] PATH_INFO: `/v3/users/2f80c00777454d60a695008760076f6e` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:13.945341 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f80c00777454d60a695008760076f6e)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:13.950940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:13.974786 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 995/1963] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:13 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:13.976263 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a0b92-9514-468c-ab63-587268548b79 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:13.982131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-129a0b92-9514-468c-ab63-587268548b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:14.018521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129a0b92-9514-468c-ab63-587268548b79 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:14.028431 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adb4b2-c15f-4c47-a61e-f94488a96481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:14.030737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adb4b2-c15f-4c47-a61e-f94488a96481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:14.031066 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adb4b2-c15f-4c47-a61e-f94488a96481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:14.031482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adb4b2-c15f-4c47-a61e-f94488a96481 admin admin] PATH_INFO: `/v3/users/699b0306f1fe4b0f93e6982797e511e1` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:14.038954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98adb4b2-c15f-4c47-a61e-f94488a96481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=699b0306f1fe4b0f93e6982797e511e1)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:14.045495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98adb4b2-c15f-4c47-a61e-f94488a96481 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:14.063213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f80c00777454d60a695008760076f6e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.064235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f80c00777454d60a695008760076f6e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.074159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f80c00777454d60a695008760076f6e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.087303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f80c00777454d60a695008760076f6e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.105589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f80c00777454d60a695008760076f6e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.115159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 969/1964] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:13 2022] DELETE /identity/v3/users/2f80c00777454d60a695008760076f6e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:14.116297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:14.116575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-98adb4b2-c15f-4c47-a61e-f94488a96481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '699b0306f1fe4b0f93e6982797e511e1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.117658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:14.130110 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-98adb4b2-c15f-4c47-a61e-f94488a96481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '699b0306f1fe4b0f93e6982797e511e1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.130916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-98adb4b2-c15f-4c47-a61e-f94488a96481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '699b0306f1fe4b0f93e6982797e511e1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.141751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-98adb4b2-c15f-4c47-a61e-f94488a96481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '699b0306f1fe4b0f93e6982797e511e1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.163689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-98adb4b2-c15f-4c47-a61e-f94488a96481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '699b0306f1fe4b0f93e6982797e511e1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.172490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 996/1965] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:13 2022] DELETE /identity/v3/users/699b0306f1fe4b0f93e6982797e511e1 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:14.174362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebe758-0fce-455a-a8f2-d4bc664dfbc8 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:14.174480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebe758-0fce-455a-a8f2-d4bc664dfbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:14.174705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebe758-0fce-455a-a8f2-d4bc664dfbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:14.219497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce2417-450f-4f33-8c46-5e1d2ef0b10f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:14.225413 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:14.240942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-70ebe758-0fce-455a-a8f2-d4bc664dfbc8 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:14.242322 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-70ebe758-0fce-455a-a8f2-d4bc664dfbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:14.341000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:14.353570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39858062-7df7-43b8-9663-4dd24029caaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:14.354325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39858062-7df7-43b8-9663-4dd24029caaa admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:14.354828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39858062-7df7-43b8-9663-4dd24029caaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:14.356097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 997/1966] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:14.359508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39858062-7df7-43b8-9663-4dd24029caaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:14.360610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39858062-7df7-43b8-9663-4dd24029caaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:14.363890 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39858062-7df7-43b8-9663-4dd24029caaa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:14.375308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebe758-0fce-455a-a8f2-d4bc664dfbc8 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:14.377138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-70ebe758-0fce-455a-a8f2-d4bc664dfbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:14.380394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 970/1967] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:14.384975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39858062-7df7-43b8-9663-4dd24029caaa admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:14.387956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-39858062-7df7-43b8-9663-4dd24029caaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:14.407268 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39858062-7df7-43b8-9663-4dd24029caaa admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:14.414367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0441f89-640e-4f98-82db-e379f1aa5f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:14.416002 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0441f89-640e-4f98-82db-e379f1aa5f0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:14.416481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0441f89-640e-4f98-82db-e379f1aa5f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:14.416901 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0441f89-640e-4f98-82db-e379f1aa5f0c admin admin] PATH_INFO: `/v3/projects/c2483add22c2473098ea696c980c6010/users/910b2df9a34c46a48caa8eea1e30e297/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:14.443226 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0441f89-640e-4f98-82db-e379f1aa5f0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2483add22c2473098ea696c980c6010, user_id=910b2df9a34c46a48caa8eea1e30e297, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:14.446645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0441f89-640e-4f98-82db-e379f1aa5f0c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:14.468879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 971/1968] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:14 2022] PUT /identity/v3/projects/c2483add22c2473098ea696c980c6010/users/910b2df9a34c46a48caa8eea1e30e297/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:25:14.473235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0441f89-640e-4f98-82db-e379f1aa5f0c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:14.491645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebe758-0fce-455a-a8f2-d4bc664dfbc8 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:14.564236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a07ae1-2b48-4743-a096-7cdc047a1f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:14.565512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a07ae1-2b48-4743-a096-7cdc047a1f30 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:14.566045 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a07ae1-2b48-4743-a096-7cdc047a1f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:14.566501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a07ae1-2b48-4743-a096-7cdc047a1f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:14.572123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0441f89-640e-4f98-82db-e379f1aa5f0c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:14.583149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a07ae1-2b48-4743-a096-7cdc047a1f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:14.587570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a07ae1-2b48-4743-a096-7cdc047a1f30 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:14.600218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb2765-4ef9-44b6-89b4-a090e68a71b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:14.600879 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb2765-4ef9-44b6-89b4-a090e68a71b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:14.601424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb2765-4ef9-44b6-89b4-a090e68a71b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:14.602103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb2765-4ef9-44b6-89b4-a090e68a71b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:14.603381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb2765-4ef9-44b6-89b4-a090e68a71b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:14.605660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb2765-4ef9-44b6-89b4-a090e68a71b1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:14.616524 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 998/1969] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:25:14.618054 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a07ae1-2b48-4743-a096-7cdc047a1f30 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:14.619544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d1a07ae1-2b48-4743-a096-7cdc047a1f30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:14.626518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 972/1970] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:14.631137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb2765-4ef9-44b6-89b4-a090e68a71b1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:14.636552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a07ae1-2b48-4743-a096-7cdc047a1f30 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:14.643559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb2765-4ef9-44b6-89b4-a090e68a71b1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:14.644324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93fb4c4-491c-4fdd-a046-b416601c4c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:14.644933 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fb4c4-491c-4fdd-a046-b416601c4c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:14.645427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fb4c4-491c-4fdd-a046-b416601c4c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:14.645809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fb4c4-491c-4fdd-a046-b416601c4c73 admin admin] PATH_INFO: `/v3/projects/d64a350643054f47b50f868c27d5ad46` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:14.647828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93fb4c4-491c-4fdd-a046-b416601c4c73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d64a350643054f47b50f868c27d5ad46)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:14.650977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93fb4c4-491c-4fdd-a046-b416601c4c73 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:14.670035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744d2fd-e42e-4daa-a6f4-a3e5345e8012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:14.670351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744d2fd-e42e-4daa-a6f4-a3e5345e8012 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:14.670729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744d2fd-e42e-4daa-a6f4-a3e5345e8012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:14.671934 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744d2fd-e42e-4daa-a6f4-a3e5345e8012 admin admin] PATH_INFO: `/v3/projects/c2483add22c2473098ea696c980c6010/users/910b2df9a34c46a48caa8eea1e30e297/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:14.677512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744d2fd-e42e-4daa-a6f4-a3e5345e8012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2483add22c2473098ea696c980c6010, user_id=910b2df9a34c46a48caa8eea1e30e297, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:14.686737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744d2fd-e42e-4daa-a6f4-a3e5345e8012 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:14.709381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 973/1971] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:14 2022] PUT /identity/v3/projects/c2483add22c2473098ea696c980c6010/users/910b2df9a34c46a48caa8eea1e30e297/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:14.721786 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f508d7-4e19-40ad-9bb8-3b1651f4f550 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:14.722299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f508d7-4e19-40ad-9bb8-3b1651f4f550 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:14.722718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f508d7-4e19-40ad-9bb8-3b1651f4f550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:14.755686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b93fb4c4-491c-4fdd-a046-b416601c4c73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd64a350643054f47b50f868c27d5ad46', 'request_id': 'req-b93fb4c4-491c-4fdd-a046-b416601c4c73'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.764021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b93fb4c4-491c-4fdd-a046-b416601c4c73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd64a350643054f47b50f868c27d5ad46', 'request_id': 'req-b93fb4c4-491c-4fdd-a046-b416601c4c73'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.783837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b93fb4c4-491c-4fdd-a046-b416601c4c73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd64a350643054f47b50f868c27d5ad46', 'request_id': 'req-b93fb4c4-491c-4fdd-a046-b416601c4c73'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.797720 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-57f508d7-4e19-40ad-9bb8-3b1651f4f550 None None] MFA Rules not processed for user `910b2df9a34c46a48caa8eea1e30e297`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:14.799248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-57f508d7-4e19-40ad-9bb8-3b1651f4f550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:14.811064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b93fb4c4-491c-4fdd-a046-b416601c4c73 admin admin] The token cache is being invalidate because project d64a350643054f47b50f868c27d5ad46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:14.811560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b93fb4c4-491c-4fdd-a046-b416601c4c73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.812435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b93fb4c4-491c-4fdd-a046-b416601c4c73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:14.815612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 999/1972] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:14 2022] DELETE /identity/v3/projects/d64a350643054f47b50f868c27d5ad46 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:14.822091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1da2af-4166-4548-9135-683f588a6eb0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:14.822618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1da2af-4166-4548-9135-683f588a6eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:14.823038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1da2af-4166-4548-9135-683f588a6eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:14.854749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ec1da2af-4166-4548-9135-683f588a6eb0 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:14.858520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ec1da2af-4166-4548-9135-683f588a6eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:14.922253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:14.922409 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:14.977671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1000/1973] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:14.979909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163de918-af1a-406d-929c-a9af19f57923 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:14.979909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163de918-af1a-406d-929c-a9af19f57923 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:14.980141 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163de918-af1a-406d-929c-a9af19f57923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:14.987249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 974/1974] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:15.002513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f508d7-4e19-40ad-9bb8-3b1651f4f550 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:15.009545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-57f508d7-4e19-40ad-9bb8-3b1651f4f550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:15.033721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-163de918-af1a-406d-929c-a9af19f57923 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:15.036360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-163de918-af1a-406d-929c-a9af19f57923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:15.057856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1001/1975] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:15.057856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163de918-af1a-406d-929c-a9af19f57923 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:15.057856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-163de918-af1a-406d-929c-a9af19f57923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:15.072610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163de918-af1a-406d-929c-a9af19f57923 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:15.077951 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:15.078980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:15.079271 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:15.079544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] PATH_INFO: `/v3/projects/61f5f14379f04984bdf40631daeac3d8` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:15.081101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61f5f14379f04984bdf40631daeac3d8)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:15.083546 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:15.113147 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f508d7-4e19-40ad-9bb8-3b1651f4f550 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:15.118208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b956a7c-8688-4299-9d56-efb9170818f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:15.118942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b956a7c-8688-4299-9d56-efb9170818f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:15.119319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b956a7c-8688-4299-9d56-efb9170818f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:15.119686 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b956a7c-8688-4299-9d56-efb9170818f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:15.137675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b956a7c-8688-4299-9d56-efb9170818f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:15.139310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b956a7c-8688-4299-9d56-efb9170818f5 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:15.152553 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 975/1976] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:15.154512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b956a7c-8688-4299-9d56-efb9170818f5 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:15.171133 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b956a7c-8688-4299-9d56-efb9170818f5 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:15.171976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61f5f14379f04984bdf40631daeac3d8', 'request_id': 'req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:15.186115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61f5f14379f04984bdf40631daeac3d8', 'request_id': 'req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:15.201862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61f5f14379f04984bdf40631daeac3d8', 'request_id': 'req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:15.226150 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] The token cache is being invalidate because project 61f5f14379f04984bdf40631daeac3d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:15.227380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:15.229583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:15.232360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1002/1977] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:15 2022] DELETE /identity/v3/projects/61f5f14379f04984bdf40631daeac3d8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:15.243730 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:15.258902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:15.279589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0bb154-f011-4c76-818d-d88978baf275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:15.281300 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0bb154-f011-4c76-818d-d88978baf275 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:15.281745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0bb154-f011-4c76-818d-d88978baf275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:15.282953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0bb154-f011-4c76-818d-d88978baf275 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:15.284439 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0bb154-f011-4c76-818d-d88978baf275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:25:15.284920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0bb154-f011-4c76-818d-d88978baf275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:15.286758 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0bb154-f011-4c76-818d-d88978baf275 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:15.317079 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 976/1978] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:25:15.322041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0bb154-f011-4c76-818d-d88978baf275 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:15.328498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7d0bb154-f011-4c76-818d-d88978baf275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:15.343888 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0bb154-f011-4c76-818d-d88978baf275 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:15.349860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371b938-58ad-4ca7-b602-38c69b90cbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:15.351823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371b938-58ad-4ca7-b602-38c69b90cbfb admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:15.352425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371b938-58ad-4ca7-b602-38c69b90cbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:15.352530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371b938-58ad-4ca7-b602-38c69b90cbfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:15.353621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371b938-58ad-4ca7-b602-38c69b90cbfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:15.357056 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371b938-58ad-4ca7-b602-38c69b90cbfb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:15.399164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06df82d-5ff4-4db8-bc99-cc89d7ea9ad6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:15.403681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 977/1979] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:15 2022] POST /identity/v3/projects => generated 385 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:15.407485 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371b938-58ad-4ca7-b602-38c69b90cbfb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:15.517467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371b938-58ad-4ca7-b602-38c69b90cbfb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:15.523183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac64ab19-453c-4d4c-9c53-47cfe859713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:15.523849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64ab19-453c-4d4c-9c53-47cfe859713c admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:15.524231 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64ab19-453c-4d4c-9c53-47cfe859713c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:15.524515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac64ab19-453c-4d4c-9c53-47cfe859713c admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:15.525256 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64ab19-453c-4d4c-9c53-47cfe859713c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:15.539441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac64ab19-453c-4d4c-9c53-47cfe859713c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:15.555470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:15.556336 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:15.556615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:15.556889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] PATH_INFO: `/v3/users/00e3e3996ccd432a80e7334b724da992` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:15.558209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00e3e3996ccd432a80e7334b724da992)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:15.560909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:15.566989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:15.568492 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:15.595065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 978/1980] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:15 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:15.598038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af4bbb-99d8-49dc-8f7b-4af1e481f584 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:15.598038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af4bbb-99d8-49dc-8f7b-4af1e481f584 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:15.598038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af4bbb-99d8-49dc-8f7b-4af1e481f584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:15.630623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-90af4bbb-99d8-49dc-8f7b-4af1e481f584 None None] MFA Rules not processed for user `9c588adcfeaf4596a69f55355d2dce08`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:15.631985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-90af4bbb-99d8-49dc-8f7b-4af1e481f584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:15.633709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00e3e3996ccd432a80e7334b724da992'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:15.645682 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00e3e3996ccd432a80e7334b724da992'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:15.646550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00e3e3996ccd432a80e7334b724da992'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:15.656835 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00e3e3996ccd432a80e7334b724da992'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:15.677151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00e3e3996ccd432a80e7334b724da992'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:15.685439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1003/1981] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:15 2022] DELETE /identity/v3/users/00e3e3996ccd432a80e7334b724da992 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:15.686798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:15.690126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:15.729485 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 979/1982] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:15 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:15.735461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af4bbb-99d8-49dc-8f7b-4af1e481f584 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:15.737773 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-90af4bbb-99d8-49dc-8f7b-4af1e481f584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:15.778513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af4bbb-99d8-49dc-8f7b-4af1e481f584 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:15.841452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a040d4b-1ed8-4fe8-ad27-0011f480fa79 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:15.853157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e62a4-e446-4205-9636-e0130be1a1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:15.855557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e62a4-e446-4205-9636-e0130be1a1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:15.857894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e62a4-e446-4205-9636-e0130be1a1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:15.860233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e62a4-e446-4205-9636-e0130be1a1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:15.860949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e62a4-e446-4205-9636-e0130be1a1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:15.868647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e62a4-e446-4205-9636-e0130be1a1d5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:15.895453 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1004/1983] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:15.899437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e62a4-e446-4205-9636-e0130be1a1d5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:15.912463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e62a4-e446-4205-9636-e0130be1a1d5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:15.920973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ae7eb-e037-4e73-a361-12b842297c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:15.921913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ae7eb-e037-4e73-a361-12b842297c09 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:15.922231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ae7eb-e037-4e73-a361-12b842297c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:15.922499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584ae7eb-e037-4e73-a361-12b842297c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:15.923268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ae7eb-e037-4e73-a361-12b842297c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:15.924826 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584ae7eb-e037-4e73-a361-12b842297c09 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:15.933511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad9a93-ccf5-45cb-8641-a7d4379a7227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:15.935806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad9a93-ccf5-45cb-8641-a7d4379a7227 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:15.936292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad9a93-ccf5-45cb-8641-a7d4379a7227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:15.936823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad9a93-ccf5-45cb-8641-a7d4379a7227 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:15.937555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad9a93-ccf5-45cb-8641-a7d4379a7227 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:15.941988 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1005/1984] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:15.947838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad9a93-ccf5-45cb-8641-a7d4379a7227 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:15.953589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ae7eb-e037-4e73-a361-12b842297c09 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:15.967642 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584ae7eb-e037-4e73-a361-12b842297c09 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:15.972888 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d51a8-cfe7-48ff-b3e5-afb82d8b1c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:15.974037 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d51a8-cfe7-48ff-b3e5-afb82d8b1c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:15.974461 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d51a8-cfe7-48ff-b3e5-afb82d8b1c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:15.974836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d51a8-cfe7-48ff-b3e5-afb82d8b1c91 admin admin] PATH_INFO: `/v3/projects/2e57f6f3102a4299a506cf0cad6dca9d/users/d92ca75287074a6082880799e8ea0ff1/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:15.997299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d51a8-cfe7-48ff-b3e5-afb82d8b1c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e57f6f3102a4299a506cf0cad6dca9d, user_id=d92ca75287074a6082880799e8ea0ff1, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:16.001404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d51a8-cfe7-48ff-b3e5-afb82d8b1c91 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:16.018736 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 980/1985] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:15 2022] POST /identity/v3/projects => generated 403 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:16.019731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad9a93-ccf5-45cb-8641-a7d4379a7227 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:16.035398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1006/1986] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:15 2022] PUT /identity/v3/projects/2e57f6f3102a4299a506cf0cad6dca9d/users/d92ca75287074a6082880799e8ea0ff1/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:16.042125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d51a8-cfe7-48ff-b3e5-afb82d8b1c91 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:16.047233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b41d51a8-cfe7-48ff-b3e5-afb82d8b1c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:16.124503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:16.141576 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad9a93-ccf5-45cb-8641-a7d4379a7227 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:16.145503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d51a8-cfe7-48ff-b3e5-afb82d8b1c91 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:16.152064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:16.156901 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5412137-74aa-4911-a0ee-033fed5bbe8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:16.157521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5412137-74aa-4911-a0ee-033fed5bbe8f service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:16.157834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5412137-74aa-4911-a0ee-033fed5bbe8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:16.158134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5412137-74aa-4911-a0ee-033fed5bbe8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:16.159939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a5412137-74aa-4911-a0ee-033fed5bbe8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:16.212708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9364e73-f64b-4807-9377-2a43e0f89d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:16.213314 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9364e73-f64b-4807-9377-2a43e0f89d02 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:16.213595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9364e73-f64b-4807-9377-2a43e0f89d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:16.213892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9364e73-f64b-4807-9377-2a43e0f89d02 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:16.214569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9364e73-f64b-4807-9377-2a43e0f89d02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:16.217071 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9364e73-f64b-4807-9377-2a43e0f89d02 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:16.234400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5412137-74aa-4911-a0ee-033fed5bbe8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:16.235766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5412137-74aa-4911-a0ee-033fed5bbe8f service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:16.240790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 981/1987] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:16 2022] POST /identity/v3/users => generated 308 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:16.240790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cffeb-4ae0-4b0b-8631-7cfc4038f25b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:16.240790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cffeb-4ae0-4b0b-8631-7cfc4038f25b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:16.240790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172cffeb-4ae0-4b0b-8631-7cfc4038f25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:16.245136 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1007/1988] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:16.245938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5412137-74aa-4911-a0ee-033fed5bbe8f service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:16.247278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a5412137-74aa-4911-a0ee-033fed5bbe8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:16.260565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5412137-74aa-4911-a0ee-033fed5bbe8f service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:16.265912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:16.266547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:16.266817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:16.267152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] PATH_INFO: `/v3/projects/16ae1fe7dc6449ac9dd662313d7a3d76` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:16.268273 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16ae1fe7dc6449ac9dd662313d7a3d76)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:16.270206 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:16.289437 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-172cffeb-4ae0-4b0b-8631-7cfc4038f25b None None] MFA Rules not processed for user `d92ca75287074a6082880799e8ea0ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:16.292382 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-172cffeb-4ae0-4b0b-8631-7cfc4038f25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:16.341700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16ae1fe7dc6449ac9dd662313d7a3d76', 'request_id': 'req-ffd7242c-e1ce-40f8-94f0-15f13134fad7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:16.349015 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16ae1fe7dc6449ac9dd662313d7a3d76', 'request_id': 'req-ffd7242c-e1ce-40f8-94f0-15f13134fad7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:16.358835 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16ae1fe7dc6449ac9dd662313d7a3d76', 'request_id': 'req-ffd7242c-e1ce-40f8-94f0-15f13134fad7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:16.377190 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] The token cache is being invalidate because project 16ae1fe7dc6449ac9dd662313d7a3d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:16.377701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:16.378677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:16.380859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1008/1989] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:16 2022] DELETE /identity/v3/projects/16ae1fe7dc6449ac9dd662313d7a3d76 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:16.386186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:16.388556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:16.403843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 982/1990] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:16.405018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d6619-d832-45ec-a5eb-a22f5349fdff None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:16.405259 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d6619-d832-45ec-a5eb-a22f5349fdff None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:16.405530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d6619-d832-45ec-a5eb-a22f5349fdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:16.430973 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c08d6619-d832-45ec-a5eb-a22f5349fdff None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:16.432900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c08d6619-d832-45ec-a5eb-a22f5349fdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:16.481963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7242c-e1ce-40f8-94f0-15f13134fad7 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:16.488352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c373a1-d741-421c-94aa-a8878b5e23a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:16.489117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c373a1-d741-421c-94aa-a8878b5e23a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:16.489660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c373a1-d741-421c-94aa-a8878b5e23a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:16.490697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c373a1-d741-421c-94aa-a8878b5e23a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:16.491636 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c373a1-d741-421c-94aa-a8878b5e23a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:16.493846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c373a1-d741-421c-94aa-a8878b5e23a2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:16.511286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1009/1991] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:16.511286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c373a1-d741-421c-94aa-a8878b5e23a2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:16.514054 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a9c373a1-d741-421c-94aa-a8878b5e23a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:16.519215 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 983/1992] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:16.523093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d6619-d832-45ec-a5eb-a22f5349fdff None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:16.534567 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d6619-d832-45ec-a5eb-a22f5349fdff None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:16.537663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:16.540843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd35fb6f-0d6f-40a0-ac7e-57cd80695cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:16.541974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35fb6f-0d6f-40a0-ac7e-57cd80695cfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:16.542418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35fb6f-0d6f-40a0-ac7e-57cd80695cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:16.542998 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd35fb6f-0d6f-40a0-ac7e-57cd80695cfa admin admin] PATH_INFO: `/v3/projects/ca5dddb2675f4cb381ad4d8ff0d32c44/users/8504971ba66242088b35028852f95f6e/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:16.565457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35fb6f-0d6f-40a0-ac7e-57cd80695cfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5dddb2675f4cb381ad4d8ff0d32c44, user_id=8504971ba66242088b35028852f95f6e, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:16.565457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd35fb6f-0d6f-40a0-ac7e-57cd80695cfa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:16.586781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 984/1993] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:16 2022] PUT /identity/v3/projects/ca5dddb2675f4cb381ad4d8ff0d32c44/users/8504971ba66242088b35028852f95f6e/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:16.588675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af784f-7ea8-468f-9060-c105a199b263 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:16.589153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af784f-7ea8-468f-9060-c105a199b263 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:16.589575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8af784f-7ea8-468f-9060-c105a199b263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:16.615133 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c8af784f-7ea8-468f-9060-c105a199b263 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:16.617740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c8af784f-7ea8-468f-9060-c105a199b263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:16.624976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:16.639803 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c373a1-d741-421c-94aa-a8878b5e23a2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:16.716598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882dee8-0c53-433a-8daa-01d02250759e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:16.717212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882dee8-0c53-433a-8daa-01d02250759e service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:16.717553 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882dee8-0c53-433a-8daa-01d02250759e service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:16.717822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882dee8-0c53-433a-8daa-01d02250759e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:16.737761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 985/1994] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:16.738585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af784f-7ea8-468f-9060-c105a199b263 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:16.751011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8af784f-7ea8-468f-9060-c105a199b263 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:16.756228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96e161-90c6-4422-abcd-906440cbc150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:16.756829 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96e161-90c6-4422-abcd-906440cbc150 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:16.757105 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96e161-90c6-4422-abcd-906440cbc150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:16.757374 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96e161-90c6-4422-abcd-906440cbc150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:16.758011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96e161-90c6-4422-abcd-906440cbc150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:16.759620 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec96e161-90c6-4422-abcd-906440cbc150 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:16.773509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 986/1995] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:16.773509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96e161-90c6-4422-abcd-906440cbc150 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:16.782800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96e161-90c6-4422-abcd-906440cbc150 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:16.785654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882dee8-0c53-433a-8daa-01d02250759e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:16.787126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882dee8-0c53-433a-8daa-01d02250759e service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:16.787550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea949481-d943-4ced-88a2-ed75c0cac7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:16.788127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea949481-d943-4ced-88a2-ed75c0cac7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:16.788402 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea949481-d943-4ced-88a2-ed75c0cac7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:16.788672 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea949481-d943-4ced-88a2-ed75c0cac7b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:16.789304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea949481-d943-4ced-88a2-ed75c0cac7b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:25:16.789601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea949481-d943-4ced-88a2-ed75c0cac7b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:16.791241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea949481-d943-4ced-88a2-ed75c0cac7b9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:16.796634 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1010/1996] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:16.797480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882dee8-0c53-433a-8daa-01d02250759e service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:16.805306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 987/1997] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:25:16.808325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882dee8-0c53-433a-8daa-01d02250759e service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:16.810019 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea949481-d943-4ced-88a2-ed75c0cac7b9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:16.814153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536d303-3f13-4892-b178-c1a936a354af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:16.814818 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536d303-3f13-4892-b178-c1a936a354af admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:16.815091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536d303-3f13-4892-b178-c1a936a354af admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:16.815434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536d303-3f13-4892-b178-c1a936a354af admin admin] PATH_INFO: `/v3/projects/ca5dddb2675f4cb381ad4d8ff0d32c44/users/8504971ba66242088b35028852f95f6e/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:16.817280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536d303-3f13-4892-b178-c1a936a354af admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5dddb2675f4cb381ad4d8ff0d32c44, user_id=8504971ba66242088b35028852f95f6e, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:16.820167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536d303-3f13-4892-b178-c1a936a354af admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:16.820844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea949481-d943-4ced-88a2-ed75c0cac7b9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:16.826667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915abe1-5682-41b1-a82b-be274ad5a426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:16.827481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915abe1-5682-41b1-a82b-be274ad5a426 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:16.827886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915abe1-5682-41b1-a82b-be274ad5a426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:16.828233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915abe1-5682-41b1-a82b-be274ad5a426 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:16.829165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915abe1-5682-41b1-a82b-be274ad5a426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:16.831272 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1011/1998] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:16 2022] PUT /identity/v3/projects/ca5dddb2675f4cb381ad4d8ff0d32c44/users/8504971ba66242088b35028852f95f6e/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:16.831759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915abe1-5682-41b1-a82b-be274ad5a426 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:16.864540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 988/1999] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:16 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:25:16.868182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536d303-3f13-4892-b178-c1a936a354af admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:16.959332 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536d303-3f13-4892-b178-c1a936a354af admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:16.982607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:17.032716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9baf5-3817-402c-a5a2-1c8fc3944aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:17.033445 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9baf5-3817-402c-a5a2-1c8fc3944aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:17.033637 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9baf5-3817-402c-a5a2-1c8fc3944aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:17.033896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9baf5-3817-402c-a5a2-1c8fc3944aa4 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:17.034582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9baf5-3817-402c-a5a2-1c8fc3944aa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:17.036740 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9baf5-3817-402c-a5a2-1c8fc3944aa4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:17.064020 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1012/2000] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:16 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:17.065467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915abe1-5682-41b1-a82b-be274ad5a426 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:17.079651 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915abe1-5682-41b1-a82b-be274ad5a426 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:17.085409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2717d1a-35c7-4a1f-98de-6fbaf06d9bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:17.086097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2717d1a-35c7-4a1f-98de-6fbaf06d9bff admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:17.086345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2717d1a-35c7-4a1f-98de-6fbaf06d9bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:17.086645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2717d1a-35c7-4a1f-98de-6fbaf06d9bff admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:17.087280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2717d1a-35c7-4a1f-98de-6fbaf06d9bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:17.088910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2717d1a-35c7-4a1f-98de-6fbaf06d9bff admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:17.100052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 989/2001] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:17.104558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9baf5-3817-402c-a5a2-1c8fc3944aa4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:17.116009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9baf5-3817-402c-a5a2-1c8fc3944aa4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:17.130418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9a0f0-43ed-4280-9318-794ced87c653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:17.131457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a0f0-43ed-4280-9318-794ced87c653 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:17.132697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a0f0-43ed-4280-9318-794ced87c653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:17.133093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a0f0-43ed-4280-9318-794ced87c653 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:17.134726 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9a0f0-43ed-4280-9318-794ced87c653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:17.136746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9a0f0-43ed-4280-9318-794ced87c653 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:17.155563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1013/2002] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:17.157008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2717d1a-35c7-4a1f-98de-6fbaf06d9bff admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:17.170911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2717d1a-35c7-4a1f-98de-6fbaf06d9bff admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:17.177455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad755447-e55a-4f6c-8872-bf6af1b7c22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:17.178800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad755447-e55a-4f6c-8872-bf6af1b7c22a admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:17.179069 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad755447-e55a-4f6c-8872-bf6af1b7c22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:17.179343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad755447-e55a-4f6c-8872-bf6af1b7c22a admin admin] PATH_INFO: `/v3/projects/50785551b3a546fe827b5bbafa9a30f1/users/796def979ed64ac79ac0f70d8393aae7/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:17.217224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad755447-e55a-4f6c-8872-bf6af1b7c22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=50785551b3a546fe827b5bbafa9a30f1, user_id=796def979ed64ac79ac0f70d8393aae7, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:17.220037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad755447-e55a-4f6c-8872-bf6af1b7c22a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:17.247730 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 990/2003] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:17 2022] PUT /identity/v3/projects/50785551b3a546fe827b5bbafa9a30f1/users/796def979ed64ac79ac0f70d8393aae7/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:17.250665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb62b76-b47b-4b75-9106-2daa123dce0c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:17.251118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb62b76-b47b-4b75-9106-2daa123dce0c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:17.251699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb62b76-b47b-4b75-9106-2daa123dce0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:17.297909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2fb62b76-b47b-4b75-9106-2daa123dce0c None None] MFA Rules not processed for user `796def979ed64ac79ac0f70d8393aae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:17.299231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2fb62b76-b47b-4b75-9106-2daa123dce0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:17.378934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1014/2004] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:17.384096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad755447-e55a-4f6c-8872-bf6af1b7c22a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:17.455997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad755447-e55a-4f6c-8872-bf6af1b7c22a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:17.461282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb12e9-b063-4475-8dd7-735a47f0f0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:17.461941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb12e9-b063-4475-8dd7-735a47f0f0f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:17.462240 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb12e9-b063-4475-8dd7-735a47f0f0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:17.462515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb12e9-b063-4475-8dd7-735a47f0f0f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:17.463381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb12e9-b063-4475-8dd7-735a47f0f0f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:17.465821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bb12e9-b063-4475-8dd7-735a47f0f0f7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:17.497821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 991/2005] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:17 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:17.502492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb62b76-b47b-4b75-9106-2daa123dce0c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:17.514687 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb62b76-b47b-4b75-9106-2daa123dce0c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:17.573591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584933cf-d3d7-482c-b55f-fade243f5f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:17.574242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584933cf-d3d7-482c-b55f-fade243f5f18 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:17.575081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584933cf-d3d7-482c-b55f-fade243f5f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:17.575352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584933cf-d3d7-482c-b55f-fade243f5f18 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:17.575987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584933cf-d3d7-482c-b55f-fade243f5f18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:17.577958 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584933cf-d3d7-482c-b55f-fade243f5f18 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:17.603666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1015/2006] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:17 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:17.605018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb12e9-b063-4475-8dd7-735a47f0f0f7 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:17.616877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb12e9-b063-4475-8dd7-735a47f0f0f7 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:17.621851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc793064-bb52-4493-8cfc-a39847075e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:17.622501 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc793064-bb52-4493-8cfc-a39847075e61 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:17.622788 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc793064-bb52-4493-8cfc-a39847075e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:17.623062 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc793064-bb52-4493-8cfc-a39847075e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:17.623712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc793064-bb52-4493-8cfc-a39847075e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:17.625348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc793064-bb52-4493-8cfc-a39847075e61 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:17.638622 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 992/2007] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:17.643096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584933cf-d3d7-482c-b55f-fade243f5f18 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:17.654753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584933cf-d3d7-482c-b55f-fade243f5f18 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:17.659870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe8e16-6857-44bc-82b4-75b1d8be7dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:17.660593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe8e16-6857-44bc-82b4-75b1d8be7dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:17.660911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe8e16-6857-44bc-82b4-75b1d8be7dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:17.661173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe8e16-6857-44bc-82b4-75b1d8be7dc8 admin admin] PATH_INFO: `/v3/projects/0af6e9f7980b4ae2b8ea80e67dc5931e/users/c123db2dab3f49b78b9fef282ce2f129/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:17.679281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe8e16-6857-44bc-82b4-75b1d8be7dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af6e9f7980b4ae2b8ea80e67dc5931e, user_id=c123db2dab3f49b78b9fef282ce2f129, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:17.682443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe8e16-6857-44bc-82b4-75b1d8be7dc8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:17.687831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:17.698935 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1016/2008] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:17 2022] PUT /identity/v3/projects/0af6e9f7980b4ae2b8ea80e67dc5931e/users/c123db2dab3f49b78b9fef282ce2f129/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:17.703427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc793064-bb52-4493-8cfc-a39847075e61 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:17.709761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:17.721099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc793064-bb52-4493-8cfc-a39847075e61 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:17.753233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d72bf-278a-42ae-8574-d787340dca27 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:17.753519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d72bf-278a-42ae-8574-d787340dca27 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:17.753706 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d72bf-278a-42ae-8574-d787340dca27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:17.783752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b18d72bf-278a-42ae-8574-d787340dca27 None None] MFA Rules not processed for user `8504971ba66242088b35028852f95f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:17.784952 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b18d72bf-278a-42ae-8574-d787340dca27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:17.804843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77db784-0ea2-42c8-9072-6c6f22e3883e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:17.806018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77db784-0ea2-42c8-9072-6c6f22e3883e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:17.806347 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77db784-0ea2-42c8-9072-6c6f22e3883e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:17.806761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77db784-0ea2-42c8-9072-6c6f22e3883e admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:17.807790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77db784-0ea2-42c8-9072-6c6f22e3883e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:17.809511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77db784-0ea2-42c8-9072-6c6f22e3883e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:17.821313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 993/2009] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:17.827436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77db784-0ea2-42c8-9072-6c6f22e3883e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:17.839242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77db784-0ea2-42c8-9072-6c6f22e3883e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:17.844942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcbae0-cd05-4eed-bc08-4daa86d15414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:17.845741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcbae0-cd05-4eed-bc08-4daa86d15414 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:17.846062 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcbae0-cd05-4eed-bc08-4daa86d15414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:17.846618 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcbae0-cd05-4eed-bc08-4daa86d15414 admin admin] PATH_INFO: `/v3/projects/0af6e9f7980b4ae2b8ea80e67dc5931e/users/c123db2dab3f49b78b9fef282ce2f129/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:17.848575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bcbae0-cd05-4eed-bc08-4daa86d15414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af6e9f7980b4ae2b8ea80e67dc5931e, user_id=c123db2dab3f49b78b9fef282ce2f129, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:17.851793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bcbae0-cd05-4eed-bc08-4daa86d15414 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:17.873811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 994/2010] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:17 2022] PUT /identity/v3/projects/0af6e9f7980b4ae2b8ea80e67dc5931e/users/c123db2dab3f49b78b9fef282ce2f129/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:17.885325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e415b-bfdb-425a-81dd-70ca0c5494bc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:17.885734 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e415b-bfdb-425a-81dd-70ca0c5494bc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:17.886145 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e415b-bfdb-425a-81dd-70ca0c5494bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:17.909766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1017/2011] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:17 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:17.929121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-da6e415b-bfdb-425a-81dd-70ca0c5494bc None None] MFA Rules not processed for user `c123db2dab3f49b78b9fef282ce2f129`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:17.930408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-da6e415b-bfdb-425a-81dd-70ca0c5494bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:18.027490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 995/2012] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:18.039711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d72bf-278a-42ae-8574-d787340dca27 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:18.102287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d72bf-278a-42ae-8574-d787340dca27 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:18.155730 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c229670-f75d-4357-8f64-5a027edf5747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:18.156331 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c229670-f75d-4357-8f64-5a027edf5747 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:18.156719 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c229670-f75d-4357-8f64-5a027edf5747 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:18.156990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c229670-f75d-4357-8f64-5a027edf5747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:18.166346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c229670-f75d-4357-8f64-5a027edf5747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:18.167767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c229670-f75d-4357-8f64-5a027edf5747 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:18.178423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1018/2013] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:18 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:18.680546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bde7cb-57ab-41a9-907f-501d0112d609 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:18.681088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bde7cb-57ab-41a9-907f-501d0112d609 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:18.681088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bde7cb-57ab-41a9-907f-501d0112d609 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:18.682638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 996/2014] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:25:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:25:18.700885 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c229670-f75d-4357-8f64-5a027edf5747 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:18.711108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd542a6f-641d-40be-b74a-769a24afdafc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:18.712025 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd542a6f-641d-40be-b74a-769a24afdafc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:18.712313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd542a6f-641d-40be-b74a-769a24afdafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:18.778115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:18.780099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-dd542a6f-641d-40be-b74a-769a24afdafc None None] MFA Rules not processed for user `3b16c430b870424c8d366c9a2959003e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:18.783720 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dd542a6f-641d-40be-b74a-769a24afdafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:18.836939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c229670-f75d-4357-8f64-5a027edf5747 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:18.844185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fe9fa-ba88-4e89-8819-6301ab91dc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:18.844986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fe9fa-ba88-4e89-8819-6301ab91dc71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:18.845267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fe9fa-ba88-4e89-8819-6301ab91dc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:18.845534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fe9fa-ba88-4e89-8819-6301ab91dc71 admin admin] PATH_INFO: `/v3/users/796def979ed64ac79ac0f70d8393aae7` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:18.846723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fe9fa-ba88-4e89-8819-6301ab91dc71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=796def979ed64ac79ac0f70d8393aae7)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:18.848696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fe9fa-ba88-4e89-8819-6301ab91dc71 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:18.925160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 997/2015] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:25:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:18.936626 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c43fe9fa-ba88-4e89-8819-6301ab91dc71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796def979ed64ac79ac0f70d8393aae7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:18.937643 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4f650-f5c9-43e6-ba95-3fc9e5d9aae4 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:18.938430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4f650-f5c9-43e6-ba95-3fc9e5d9aae4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:18.938962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4f650-f5c9-43e6-ba95-3fc9e5d9aae4 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:18.941906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 998/2016] 10.222.0.55 () {54 vars in 961 bytes} [Tue Sep 27 17:25:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 27 17:25:18.953897 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289aa2e8-05f8-4154-8ce4-e2bde2b188fc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:18.954740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289aa2e8-05f8-4154-8ce4-e2bde2b188fc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:18.957127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289aa2e8-05f8-4154-8ce4-e2bde2b188fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:18.977418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c43fe9fa-ba88-4e89-8819-6301ab91dc71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796def979ed64ac79ac0f70d8393aae7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:18.978703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c43fe9fa-ba88-4e89-8819-6301ab91dc71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796def979ed64ac79ac0f70d8393aae7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:18.993087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:19.006211 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-289aa2e8-05f8-4154-8ce4-e2bde2b188fc None None] MFA Rules not processed for user `3b16c430b870424c8d366c9a2959003e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:19.007105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c43fe9fa-ba88-4e89-8819-6301ab91dc71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796def979ed64ac79ac0f70d8393aae7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:19.009749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-289aa2e8-05f8-4154-8ce4-e2bde2b188fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:19.034799 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 999/2017] 10.222.0.55 () {58 vars in 1058 bytes} [Tue Sep 27 17:25:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:19.044092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289aa2e8-05f8-4154-8ce4-e2bde2b188fc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:19.052281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c43fe9fa-ba88-4e89-8819-6301ab91dc71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796def979ed64ac79ac0f70d8393aae7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:19.059504 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1019/2018] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:18 2022] DELETE /identity/v3/users/796def979ed64ac79ac0f70d8393aae7 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:19.065617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289aa2e8-05f8-4154-8ce4-e2bde2b188fc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:19.084518 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89814b9b-49dc-4c8e-a494-9d9e3892f61d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:19.084922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89814b9b-49dc-4c8e-a494-9d9e3892f61d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:19.085297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89814b9b-49dc-4c8e-a494-9d9e3892f61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:19.131070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-89814b9b-49dc-4c8e-a494-9d9e3892f61d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:19.132549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-89814b9b-49dc-4c8e-a494-9d9e3892f61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:19.179318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9180d6d-9948-4bcc-8183-83ab536ccddb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:19.180520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9180d6d-9948-4bcc-8183-83ab536ccddb service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:19.181054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9180d6d-9948-4bcc-8183-83ab536ccddb service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:19.182143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9180d6d-9948-4bcc-8183-83ab536ccddb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:19.196130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c9180d6d-9948-4bcc-8183-83ab536ccddb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:19.220223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9180d6d-9948-4bcc-8183-83ab536ccddb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:19.221864 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9180d6d-9948-4bcc-8183-83ab536ccddb service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:19.240724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1000/2019] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:25:19.247902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1020/2020] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:19.261067 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9180d6d-9948-4bcc-8183-83ab536ccddb service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:19.265063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c9180d6d-9948-4bcc-8183-83ab536ccddb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:19.350676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9180d6d-9948-4bcc-8183-83ab536ccddb service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:19.356924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167b8eb-5710-41f2-a7fa-d1cb367fc5f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:19.357595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167b8eb-5710-41f2-a7fa-d1cb367fc5f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:19.358257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167b8eb-5710-41f2-a7fa-d1cb367fc5f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:19.358598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167b8eb-5710-41f2-a7fa-d1cb367fc5f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:19.372479 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167b8eb-5710-41f2-a7fa-d1cb367fc5f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:19.374942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167b8eb-5710-41f2-a7fa-d1cb367fc5f3 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:19.404039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1001/2021] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:20.046322 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89814b9b-49dc-4c8e-a494-9d9e3892f61d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:20.047900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-89814b9b-49dc-4c8e-a494-9d9e3892f61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:20.060434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89814b9b-49dc-4c8e-a494-9d9e3892f61d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:20.068492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:20.069153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:20.069449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:20.069739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] PATH_INFO: `/v3/users/9c588adcfeaf4596a69f55355d2dce08` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:20.070938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c588adcfeaf4596a69f55355d2dce08)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:20.072937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:20.131946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c588adcfeaf4596a69f55355d2dce08'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.136599 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167b8eb-5710-41f2-a7fa-d1cb367fc5f3 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:20.139512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d167b8eb-5710-41f2-a7fa-d1cb367fc5f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:20.140537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:20.143477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c588adcfeaf4596a69f55355d2dce08'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.144466 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c588adcfeaf4596a69f55355d2dce08'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.156187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167b8eb-5710-41f2-a7fa-d1cb367fc5f3 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:20.161713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:20.162487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:20.162812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:20.163252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] PATH_INFO: `/v3/projects/50785551b3a546fe827b5bbafa9a30f1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:20.164702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50785551b3a546fe827b5bbafa9a30f1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:20.166937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:20.176252 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c588adcfeaf4596a69f55355d2dce08'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.197827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c588adcfeaf4596a69f55355d2dce08'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.205109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1021/2022] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:20 2022] DELETE /identity/v3/users/9c588adcfeaf4596a69f55355d2dce08 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:20.257922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50785551b3a546fe827b5bbafa9a30f1', 'request_id': 'req-1f402583-0e79-48e3-9eea-8bdabe385e89'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.264740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50785551b3a546fe827b5bbafa9a30f1', 'request_id': 'req-1f402583-0e79-48e3-9eea-8bdabe385e89'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.273796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50785551b3a546fe827b5bbafa9a30f1', 'request_id': 'req-1f402583-0e79-48e3-9eea-8bdabe385e89'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.291402 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] The token cache is being invalidate because project 50785551b3a546fe827b5bbafa9a30f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:20.291694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.292426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.294583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1002/2023] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:20 2022] DELETE /identity/v3/projects/50785551b3a546fe827b5bbafa9a30f1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:20.298632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:20.300178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:20.401505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c429671-4b1a-4a5e-8b93-465ba67c3f7e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:20.469084 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:20.470556 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:20.518267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:20.519060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:20.520167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:20.520167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] PATH_INFO: `/v3/users/c123db2dab3f49b78b9fef282ce2f129` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:20.520951 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c123db2dab3f49b78b9fef282ce2f129)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:20.521441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:20.524338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:20.582824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f402583-0e79-48e3-9eea-8bdabe385e89 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:20.587600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52239d35-1749-4641-8814-14edad265558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:20.588225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52239d35-1749-4641-8814-14edad265558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:20.588502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52239d35-1749-4641-8814-14edad265558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:20.588617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c123db2dab3f49b78b9fef282ce2f129'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.588771 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52239d35-1749-4641-8814-14edad265558 admin admin] PATH_INFO: `/v3/projects/3ddac4f035c44df5a409c0ed272ee4cf` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:20.589893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52239d35-1749-4641-8814-14edad265558 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ddac4f035c44df5a409c0ed272ee4cf)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:20.592537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52239d35-1749-4641-8814-14edad265558 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:20.597356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c123db2dab3f49b78b9fef282ce2f129'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.598040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c123db2dab3f49b78b9fef282ce2f129'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.606738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c123db2dab3f49b78b9fef282ce2f129'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.620477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c123db2dab3f49b78b9fef282ce2f129'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.622370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:20.630231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1022/2024] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:20 2022] DELETE /identity/v3/users/c123db2dab3f49b78b9fef282ce2f129 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:25:20.651190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-52239d35-1749-4641-8814-14edad265558 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ddac4f035c44df5a409c0ed272ee4cf', 'request_id': 'req-52239d35-1749-4641-8814-14edad265558'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.657869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-52239d35-1749-4641-8814-14edad265558 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ddac4f035c44df5a409c0ed272ee4cf', 'request_id': 'req-52239d35-1749-4641-8814-14edad265558'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.665395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-52239d35-1749-4641-8814-14edad265558 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ddac4f035c44df5a409c0ed272ee4cf', 'request_id': 'req-52239d35-1749-4641-8814-14edad265558'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.679946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-52239d35-1749-4641-8814-14edad265558 admin admin] The token cache is being invalidate because project 3ddac4f035c44df5a409c0ed272ee4cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:20.680255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-52239d35-1749-4641-8814-14edad265558 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.681022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-52239d35-1749-4641-8814-14edad265558 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.682598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1003/2025] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:20 2022] DELETE /identity/v3/projects/3ddac4f035c44df5a409c0ed272ee4cf => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:20.687400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:20.688821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:20.733668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52239d35-1749-4641-8814-14edad265558 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:20.735217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-52239d35-1749-4641-8814-14edad265558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:20.784608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a301b15-2caa-4437-a58b-6d2a427aa752 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:20.816209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52239d35-1749-4641-8814-14edad265558 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:20.823568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd36292e-d511-44c8-ad36-75912e49c850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:20.824378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36292e-d511-44c8-ad36-75912e49c850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:20.824722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36292e-d511-44c8-ad36-75912e49c850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:20.825091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd36292e-d511-44c8-ad36-75912e49c850 admin admin] PATH_INFO: `/v3/users/d92ca75287074a6082880799e8ea0ff1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:20.826445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36292e-d511-44c8-ad36-75912e49c850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d92ca75287074a6082880799e8ea0ff1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:20.828823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd36292e-d511-44c8-ad36-75912e49c850 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:20.865701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:20.866557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:20.867038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:20.867276 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] PATH_INFO: `/v3/users/8504971ba66242088b35028852f95f6e` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:20.868843 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8504971ba66242088b35028852f95f6e)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:20.871018 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:20.911062 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd36292e-d511-44c8-ad36-75912e49c850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd92ca75287074a6082880799e8ea0ff1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.911746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd36292e-d511-44c8-ad36-75912e49c850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd92ca75287074a6082880799e8ea0ff1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.913239 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:20.928230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd36292e-d511-44c8-ad36-75912e49c850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd92ca75287074a6082880799e8ea0ff1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.938660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd36292e-d511-44c8-ad36-75912e49c850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd92ca75287074a6082880799e8ea0ff1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.948625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8504971ba66242088b35028852f95f6e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.953724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd36292e-d511-44c8-ad36-75912e49c850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd92ca75287074a6082880799e8ea0ff1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.957648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8504971ba66242088b35028852f95f6e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.958370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8504971ba66242088b35028852f95f6e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.961266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1004/2026] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:20 2022] DELETE /identity/v3/users/d92ca75287074a6082880799e8ea0ff1 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:25:20.966541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ef8e7-a322-48a2-9901-6511a3ec360e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:20.966829 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ef8e7-a322-48a2-9901-6511a3ec360e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:20.967356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ef8e7-a322-48a2-9901-6511a3ec360e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:20.969233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8504971ba66242088b35028852f95f6e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.985515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8504971ba66242088b35028852f95f6e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:20.989462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ba8ef8e7-a322-48a2-9901-6511a3ec360e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:20.990731 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ba8ef8e7-a322-48a2-9901-6511a3ec360e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:20.992522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1023/2027] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:20 2022] DELETE /identity/v3/users/8504971ba66242088b35028852f95f6e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:25:21.078969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1005/2028] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:21.090720 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:21.092890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:21.134095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:21.197905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d20dd-8cfe-450a-9735-a5702cd141fd admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:21.203299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f2fb5-c4b0-498a-9676-f9289ecb2e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:21.204049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f2fb5-c4b0-498a-9676-f9289ecb2e12 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:21.204432 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f2fb5-c4b0-498a-9676-f9289ecb2e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:21.204804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f2fb5-c4b0-498a-9676-f9289ecb2e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:21.220207 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f2fb5-c4b0-498a-9676-f9289ecb2e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:21.222613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177f2fb5-c4b0-498a-9676-f9289ecb2e12 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:21.241749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1024/2029] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:21.351808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ef8e7-a322-48a2-9901-6511a3ec360e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:21.353366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ba8ef8e7-a322-48a2-9901-6511a3ec360e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:21.421698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ef8e7-a322-48a2-9901-6511a3ec360e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:21.448548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85cc55-18c2-42ee-8113-71565164d4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:21.449735 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cc55-18c2-42ee-8113-71565164d4b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:21.450039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cc55-18c2-42ee-8113-71565164d4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:21.450338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cc55-18c2-42ee-8113-71565164d4b1 admin admin] PATH_INFO: `/v3/projects/ca5dddb2675f4cb381ad4d8ff0d32c44` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:21.454780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85cc55-18c2-42ee-8113-71565164d4b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5dddb2675f4cb381ad4d8ff0d32c44)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:21.456806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85cc55-18c2-42ee-8113-71565164d4b1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:21.507986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f2fb5-c4b0-498a-9676-f9289ecb2e12 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:21.510327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-177f2fb5-c4b0-498a-9676-f9289ecb2e12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:21.534847 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f2fb5-c4b0-498a-9676-f9289ecb2e12 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:21.557109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bf85cc55-18c2-42ee-8113-71565164d4b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5dddb2675f4cb381ad4d8ff0d32c44', 'request_id': 'req-bf85cc55-18c2-42ee-8113-71565164d4b1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:21.565660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bf85cc55-18c2-42ee-8113-71565164d4b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5dddb2675f4cb381ad4d8ff0d32c44', 'request_id': 'req-bf85cc55-18c2-42ee-8113-71565164d4b1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:21.573408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bf85cc55-18c2-42ee-8113-71565164d4b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5dddb2675f4cb381ad4d8ff0d32c44', 'request_id': 'req-bf85cc55-18c2-42ee-8113-71565164d4b1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:21.614110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bf85cc55-18c2-42ee-8113-71565164d4b1 admin admin] The token cache is being invalidate because project ca5dddb2675f4cb381ad4d8ff0d32c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:21.614110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bf85cc55-18c2-42ee-8113-71565164d4b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:21.616532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bf85cc55-18c2-42ee-8113-71565164d4b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:21.622587 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1006/2030] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:21 2022] DELETE /identity/v3/projects/ca5dddb2675f4cb381ad4d8ff0d32c44 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:21.627322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c732d97-2c4c-485c-a189-328360910018 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:21.627736 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c732d97-2c4c-485c-a189-328360910018 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:21.628103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c732d97-2c4c-485c-a189-328360910018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:21.673549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:21.674255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:21.674621 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:21.675119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] PATH_INFO: `/v3/projects/0af6e9f7980b4ae2b8ea80e67dc5931e` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:21.676408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af6e9f7980b4ae2b8ea80e67dc5931e)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:21.678440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:21.682095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6c732d97-2c4c-485c-a189-328360910018 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:21.687322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6c732d97-2c4c-485c-a189-328360910018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:21.703485 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1007/2031] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:21.711128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9a4de-6c31-4b59-8dee-7c128421085e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:21.711475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9a4de-6c31-4b59-8dee-7c128421085e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:21.711748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9a4de-6c31-4b59-8dee-7c128421085e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:21.742337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f5b9a4de-6c31-4b59-8dee-7c128421085e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:21.743715 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f5b9a4de-6c31-4b59-8dee-7c128421085e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:21.768121 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af6e9f7980b4ae2b8ea80e67dc5931e', 'request_id': 'req-067e76c1-637d-45e8-ab3b-d44e87385bf5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:21.774340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af6e9f7980b4ae2b8ea80e67dc5931e', 'request_id': 'req-067e76c1-637d-45e8-ab3b-d44e87385bf5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:21.783376 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af6e9f7980b4ae2b8ea80e67dc5931e', 'request_id': 'req-067e76c1-637d-45e8-ab3b-d44e87385bf5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:21.807206 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] The token cache is being invalidate because project 0af6e9f7980b4ae2b8ea80e67dc5931e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:21.807525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:21.808171 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:21.810265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1025/2032] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:21 2022] DELETE /identity/v3/projects/0af6e9f7980b4ae2b8ea80e67dc5931e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:25:21.814458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:21.814821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:21.815938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:21.840979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1008/2033] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:21.842404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bc631-d6d5-40f0-87db-cf3f96493dc7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:21.842736 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bc631-d6d5-40f0-87db-cf3f96493dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:21.843016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810bc631-d6d5-40f0-87db-cf3f96493dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:21.869276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-810bc631-d6d5-40f0-87db-cf3f96493dc7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:21.870544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-810bc631-d6d5-40f0-87db-cf3f96493dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:21.879693 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1009/2034] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:21.880567 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bc631-d6d5-40f0-87db-cf3f96493dc7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:21.893125 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810bc631-d6d5-40f0-87db-cf3f96493dc7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:21.897614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:21.900436 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e76c1-637d-45e8-ab3b-d44e87385bf5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:21.919228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:21.920580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:21.921281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:21.921528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e997a6-6d93-4124-a0b8-db5a73a15860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:21.921928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] PATH_INFO: `/v3/projects/2e57f6f3102a4299a506cf0cad6dca9d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:21.922212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e997a6-6d93-4124-a0b8-db5a73a15860 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:21.922530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e997a6-6d93-4124-a0b8-db5a73a15860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:21.922826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e997a6-6d93-4124-a0b8-db5a73a15860 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:21.923544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e997a6-6d93-4124-a0b8-db5a73a15860 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:25:21.923807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e997a6-6d93-4124-a0b8-db5a73a15860 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:21.924574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e57f6f3102a4299a506cf0cad6dca9d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:21.927049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:21.929909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e997a6-6d93-4124-a0b8-db5a73a15860 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:21.954513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1010/2035] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:25:21.954513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bd521-a2fc-4abd-8f21-ab4be20524e4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:21.954513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bd521-a2fc-4abd-8f21-ab4be20524e4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:21.954513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bd521-a2fc-4abd-8f21-ab4be20524e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:21.976418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-df7bd521-a2fc-4abd-8f21-ab4be20524e4 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:21.977640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-df7bd521-a2fc-4abd-8f21-ab4be20524e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:21.989890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e57f6f3102a4299a506cf0cad6dca9d', 'request_id': 'req-0fc65d69-9336-45bd-9ad8-5cd454922939'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:21.997781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e57f6f3102a4299a506cf0cad6dca9d', 'request_id': 'req-0fc65d69-9336-45bd-9ad8-5cd454922939'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:22.007281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e57f6f3102a4299a506cf0cad6dca9d', 'request_id': 'req-0fc65d69-9336-45bd-9ad8-5cd454922939'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:22.029528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] The token cache is being invalidate because project 2e57f6f3102a4299a506cf0cad6dca9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:22.029937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:22.030695 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:22.032734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1026/2036] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:21 2022] DELETE /identity/v3/projects/2e57f6f3102a4299a506cf0cad6dca9d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:22.033502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.034964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:22.054856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1011/2037] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:22.056565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79f14b-985f-4332-8c9c-4bbb5054f2be None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.057033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79f14b-985f-4332-8c9c-4bbb5054f2be None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.057602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79f14b-985f-4332-8c9c-4bbb5054f2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.084210 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6e79f14b-985f-4332-8c9c-4bbb5054f2be None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:22.086128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6e79f14b-985f-4332-8c9c-4bbb5054f2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:22.111181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1012/2038] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:22.112121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79f14b-985f-4332-8c9c-4bbb5054f2be None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.131988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79f14b-985f-4332-8c9c-4bbb5054f2be None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.137586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78da588-6e79-42f3-954a-808fed5c715d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.138475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78da588-6e79-42f3-954a-808fed5c715d admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.138978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78da588-6e79-42f3-954a-808fed5c715d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.139444 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78da588-6e79-42f3-954a-808fed5c715d admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.140024 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc65d69-9336-45bd-9ad8-5cd454922939 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.141126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78da588-6e79-42f3-954a-808fed5c715d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:25:22.141617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78da588-6e79-42f3-954a-808fed5c715d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.143463 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78da588-6e79-42f3-954a-808fed5c715d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.147387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89a2ce-34cd-4722-81da-4ba8e17616cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.148504 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89a2ce-34cd-4722-81da-4ba8e17616cd admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.149109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89a2ce-34cd-4722-81da-4ba8e17616cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.149603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89a2ce-34cd-4722-81da-4ba8e17616cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.150760 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89a2ce-34cd-4722-81da-4ba8e17616cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.153995 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89a2ce-34cd-4722-81da-4ba8e17616cd admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.160752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1013/2039] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:25:22.162655 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a51d9-3f79-437a-bd79-b959491c0820 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.163122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a51d9-3f79-437a-bd79-b959491c0820 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.163606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a51d9-3f79-437a-bd79-b959491c0820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.199131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-938a51d9-3f79-437a-bd79-b959491c0820 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:22.201934 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-938a51d9-3f79-437a-bd79-b959491c0820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:22.206143 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1027/2040] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:22 2022] POST /identity/v3/projects => generated 371 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:22.211985 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89a2ce-34cd-4722-81da-4ba8e17616cd admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.287500 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1014/2041] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:22.287500 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a51d9-3f79-437a-bd79-b959491c0820 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.295632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a51d9-3f79-437a-bd79-b959491c0820 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.301329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31216d05-8677-457e-aaab-d80eb9fc4ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.301989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31216d05-8677-457e-aaab-d80eb9fc4ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.302296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31216d05-8677-457e-aaab-d80eb9fc4ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.302568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31216d05-8677-457e-aaab-d80eb9fc4ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.303213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31216d05-8677-457e-aaab-d80eb9fc4ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.305458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31216d05-8677-457e-aaab-d80eb9fc4ae0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.307816 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89a2ce-34cd-4722-81da-4ba8e17616cd admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.322124 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b24108-6bda-4fe1-bb1d-f4679cd687e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.322844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b24108-6bda-4fe1-bb1d-f4679cd687e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.323131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b24108-6bda-4fe1-bb1d-f4679cd687e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.323424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b24108-6bda-4fe1-bb1d-f4679cd687e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.324783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b24108-6bda-4fe1-bb1d-f4679cd687e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.326827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b24108-6bda-4fe1-bb1d-f4679cd687e6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.334523 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1015/2042] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:22 2022] POST /identity/v3/users => generated 286 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:22.335724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31216d05-8677-457e-aaab-d80eb9fc4ae0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.350516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31216d05-8677-457e-aaab-d80eb9fc4ae0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.356848 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff99d4-d345-4af3-88b0-8c4655b139f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.357609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:22.358794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff99d4-d345-4af3-88b0-8c4655b139f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.359572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff99d4-d345-4af3-88b0-8c4655b139f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.360039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff99d4-d345-4af3-88b0-8c4655b139f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.360888 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff99d4-d345-4af3-88b0-8c4655b139f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:25:22.361316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff99d4-d345-4af3-88b0-8c4655b139f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.363817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff99d4-d345-4af3-88b0-8c4655b139f6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.378935 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1028/2043] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:22 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:22.378935 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b24108-6bda-4fe1-bb1d-f4679cd687e6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.384158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1016/2044] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:25:22.384158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff99d4-d345-4af3-88b0-8c4655b139f6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.393491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff99d4-d345-4af3-88b0-8c4655b139f6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.410221 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:22.478129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b24108-6bda-4fe1-bb1d-f4679cd687e6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.484217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782a4c4-5fe6-4f80-ad2c-79b5465cbdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.484779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be273ed-9dbf-4451-900d-b89f6732eb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.484987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782a4c4-5fe6-4f80-ad2c-79b5465cbdea admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.485166 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782a4c4-5fe6-4f80-ad2c-79b5465cbdea admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.485449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782a4c4-5fe6-4f80-ad2c-79b5465cbdea admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.485768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be273ed-9dbf-4451-900d-b89f6732eb33 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.486107 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782a4c4-5fe6-4f80-ad2c-79b5465cbdea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.486280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be273ed-9dbf-4451-900d-b89f6732eb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.486596 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be273ed-9dbf-4451-900d-b89f6732eb33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.487690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be273ed-9dbf-4451-900d-b89f6732eb33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.488518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782a4c4-5fe6-4f80-ad2c-79b5465cbdea admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.489984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be273ed-9dbf-4451-900d-b89f6732eb33 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.506302 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1029/2045] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:22.507390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be273ed-9dbf-4451-900d-b89f6732eb33 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.519106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be273ed-9dbf-4451-900d-b89f6732eb33 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.526974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95773976-a33b-4bae-879b-3cc28d9dceb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.527866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95773976-a33b-4bae-879b-3cc28d9dceb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.528315 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95773976-a33b-4bae-879b-3cc28d9dceb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.528757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95773976-a33b-4bae-879b-3cc28d9dceb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.529540 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95773976-a33b-4bae-879b-3cc28d9dceb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.532330 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95773976-a33b-4bae-879b-3cc28d9dceb5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.533457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1017/2046] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:22 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:22.535784 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782a4c4-5fe6-4f80-ad2c-79b5465cbdea admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.549413 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782a4c4-5fe6-4f80-ad2c-79b5465cbdea admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.570768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bf695-61f2-4003-ab1e-19b99e0445f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.571499 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bf695-61f2-4003-ab1e-19b99e0445f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.572142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bf695-61f2-4003-ab1e-19b99e0445f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.572494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bf695-61f2-4003-ab1e-19b99e0445f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.573218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bf695-61f2-4003-ab1e-19b99e0445f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.575399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bf695-61f2-4003-ab1e-19b99e0445f5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.586007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1030/2047] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:22 2022] POST /identity/v3/projects => generated 379 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:22.586007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95773976-a33b-4bae-879b-3cc28d9dceb5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.587144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1018/2048] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:22.587968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bf695-61f2-4003-ab1e-19b99e0445f5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.617923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bf695-61f2-4003-ab1e-19b99e0445f5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.684000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95773976-a33b-4bae-879b-3cc28d9dceb5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.689385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727e967-2d5b-4544-a9bc-da6f113592d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.689993 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727e967-2d5b-4544-a9bc-da6f113592d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.690317 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727e967-2d5b-4544-a9bc-da6f113592d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.690594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727e967-2d5b-4544-a9bc-da6f113592d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.691247 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727e967-2d5b-4544-a9bc-da6f113592d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.692849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a05ec-5459-436f-b09d-ba25f851322a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.692995 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b727e967-2d5b-4544-a9bc-da6f113592d6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.693573 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a05ec-5459-436f-b09d-ba25f851322a admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.694010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a05ec-5459-436f-b09d-ba25f851322a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.694442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a05ec-5459-436f-b09d-ba25f851322a admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.695455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a05ec-5459-436f-b09d-ba25f851322a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.699717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a05ec-5459-436f-b09d-ba25f851322a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.704615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1031/2049] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:22.705425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727e967-2d5b-4544-a9bc-da6f113592d6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.722853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727e967-2d5b-4544-a9bc-da6f113592d6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.729671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ecc18-d6fe-47c0-829b-0ef24799b251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.731430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ecc18-d6fe-47c0-829b-0ef24799b251 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.731774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ecc18-d6fe-47c0-829b-0ef24799b251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.732797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ecc18-d6fe-47c0-829b-0ef24799b251 admin admin] PATH_INFO: `/v3/projects/1fcdacccb639490ba9986dace34a60cc/users/7417a079de114575840c6d3f1c9716e0/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.741100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1019/2050] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:22 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:22.741100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a05ec-5459-436f-b09d-ba25f851322a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.756764 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a05ec-5459-436f-b09d-ba25f851322a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.762918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb92e3-765d-47f3-b8d7-33c55a025ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.763555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb92e3-765d-47f3-b8d7-33c55a025ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.764178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb92e3-765d-47f3-b8d7-33c55a025ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.764178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb92e3-765d-47f3-b8d7-33c55a025ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.764915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb92e3-765d-47f3-b8d7-33c55a025ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.765305 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ecc18-d6fe-47c0-829b-0ef24799b251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fcdacccb639490ba9986dace34a60cc, user_id=7417a079de114575840c6d3f1c9716e0, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.766702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb92e3-765d-47f3-b8d7-33c55a025ee3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.769855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ecc18-d6fe-47c0-829b-0ef24799b251 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.782730 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1020/2051] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:22.782730 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb92e3-765d-47f3-b8d7-33c55a025ee3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.797468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1032/2052] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:22 2022] PUT /identity/v3/projects/1fcdacccb639490ba9986dace34a60cc/users/7417a079de114575840c6d3f1c9716e0/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:22.798407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ecc18-d6fe-47c0-829b-0ef24799b251 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:22.807216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb92e3-765d-47f3-b8d7-33c55a025ee3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.910751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:22.915875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf402f-200a-46f9-8b23-b4c3ec4df6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.916534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf402f-200a-46f9-8b23-b4c3ec4df6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.917170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf402f-200a-46f9-8b23-b4c3ec4df6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.919579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf402f-200a-46f9-8b23-b4c3ec4df6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.920789 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf402f-200a-46f9-8b23-b4c3ec4df6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.923149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf402f-200a-46f9-8b23-b4c3ec4df6a9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.938616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ecc18-d6fe-47c0-829b-0ef24799b251 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:22.941686 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1021/2053] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:22.943101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd64fa7-e593-46be-9e5a-33c3c22cc226 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.943389 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd64fa7-e593-46be-9e5a-33c3c22cc226 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.943678 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd64fa7-e593-46be-9e5a-33c3c22cc226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.949034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5722d7-ee97-4313-a688-874bb0c9c5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:22.950165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5722d7-ee97-4313-a688-874bb0c9c5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:22.950459 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5722d7-ee97-4313-a688-874bb0c9c5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:22.950737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5722d7-ee97-4313-a688-874bb0c9c5bc admin admin] PATH_INFO: `/v3/projects/f20f02f037ee463f9cd93ad3eece56ba/users/611ff79435f04fa198640f599a254076/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:22.960855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:22.971066 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5722d7-ee97-4313-a688-874bb0c9c5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20f02f037ee463f9cd93ad3eece56ba, user_id=611ff79435f04fa198640f599a254076, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:22.973758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5722d7-ee97-4313-a688-874bb0c9c5bc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:22.984046 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ebd64fa7-e593-46be-9e5a-33c3c22cc226 None None] MFA Rules not processed for user `7417a079de114575840c6d3f1c9716e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:22.986199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ebd64fa7-e593-46be-9e5a-33c3c22cc226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:22.994112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1033/2054] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:22 2022] PUT /identity/v3/projects/f20f02f037ee463f9cd93ad3eece56ba/users/611ff79435f04fa198640f599a254076/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:22.994112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5722d7-ee97-4313-a688-874bb0c9c5bc admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:23.086364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1022/2055] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:23.088109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60e510-04be-4919-844e-6c60e3353f42 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.088397 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60e510-04be-4919-844e-6c60e3353f42 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.088718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60e510-04be-4919-844e-6c60e3353f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.095020 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5722d7-ee97-4313-a688-874bb0c9c5bc admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:23.103139 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de00502-c688-4d77-9d9c-0d2360fedc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:23.104668 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de00502-c688-4d77-9d9c-0d2360fedc57 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.105196 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de00502-c688-4d77-9d9c-0d2360fedc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.106459 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de00502-c688-4d77-9d9c-0d2360fedc57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.108814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de00502-c688-4d77-9d9c-0d2360fedc57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:23.111907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de00502-c688-4d77-9d9c-0d2360fedc57 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:23.125631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-cb60e510-04be-4919-844e-6c60e3353f42 None None] MFA Rules not processed for user `611ff79435f04fa198640f599a254076`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:23.127458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cb60e510-04be-4919-844e-6c60e3353f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:23.137075 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1034/2056] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:23.137075 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de00502-c688-4d77-9d9c-0d2360fedc57 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:23.139158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3de00502-c688-4d77-9d9c-0d2360fedc57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:23.212426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de00502-c688-4d77-9d9c-0d2360fedc57 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:23.218681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df1578-82b7-48d4-9d1a-f60f19e8a941 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:23.219914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df1578-82b7-48d4-9d1a-f60f19e8a941 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.220309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df1578-82b7-48d4-9d1a-f60f19e8a941 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.220548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df1578-82b7-48d4-9d1a-f60f19e8a941 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.222878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1023/2057] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:23.226288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-19df1578-82b7-48d4-9d1a-f60f19e8a941 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:23.226411 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60e510-04be-4919-844e-6c60e3353f42 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:23.236789 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60e510-04be-4919-844e-6c60e3353f42 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:23.244571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb349e-b0bf-4c61-b794-852afba0c61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:23.244571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb349e-b0bf-4c61-b794-852afba0c61c admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.244571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb349e-b0bf-4c61-b794-852afba0c61c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.244571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb349e-b0bf-4c61-b794-852afba0c61c admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.244983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb349e-b0bf-4c61-b794-852afba0c61c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:23.246266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb349e-b0bf-4c61-b794-852afba0c61c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:23.281364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1024/2058] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:23 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:23.282616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb349e-b0bf-4c61-b794-852afba0c61c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:23.298529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df1578-82b7-48d4-9d1a-f60f19e8a941 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:23.300011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df1578-82b7-48d4-9d1a-f60f19e8a941 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:23.317409 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1035/2059] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:23 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:23.318209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df1578-82b7-48d4-9d1a-f60f19e8a941 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:23.330296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df1578-82b7-48d4-9d1a-f60f19e8a941 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:23.344475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:23.368639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb349e-b0bf-4c61-b794-852afba0c61c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:23.374355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedc7b9-46d3-46ea-a110-f011993a0fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:23.375119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedc7b9-46d3-46ea-a110-f011993a0fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.375506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedc7b9-46d3-46ea-a110-f011993a0fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.375936 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedc7b9-46d3-46ea-a110-f011993a0fb0 admin admin] PATH_INFO: `/v3/projects/36351a4bc84a45cb866e72f52e6e725a/users/1dba953d778c418abea22efac408e2db/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.394417 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:23.412156 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedc7b9-46d3-46ea-a110-f011993a0fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36351a4bc84a45cb866e72f52e6e725a, user_id=1dba953d778c418abea22efac408e2db, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:23.415227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedc7b9-46d3-46ea-a110-f011993a0fb0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:23.423770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d94207-2d35-4945-b200-3c06b3424dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:23.424618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d94207-2d35-4945-b200-3c06b3424dda admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.424912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d94207-2d35-4945-b200-3c06b3424dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.425207 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d94207-2d35-4945-b200-3c06b3424dda admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.428161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d94207-2d35-4945-b200-3c06b3424dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:23.430215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d94207-2d35-4945-b200-3c06b3424dda admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:23.439238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1025/2060] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:23 2022] PUT /identity/v3/projects/36351a4bc84a45cb866e72f52e6e725a/users/1dba953d778c418abea22efac408e2db/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:23.443144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedc7b9-46d3-46ea-a110-f011993a0fb0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:23.449015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4cedc7b9-46d3-46ea-a110-f011993a0fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:23.476949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1036/2061] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:23 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:23.476949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3bddc-8e31-486e-806f-038bb1a9e177 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.477193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3bddc-8e31-486e-806f-038bb1a9e177 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.477309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3bddc-8e31-486e-806f-038bb1a9e177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.511338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-21b3bddc-8e31-486e-806f-038bb1a9e177 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:23.513248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-21b3bddc-8e31-486e-806f-038bb1a9e177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:23.546757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedc7b9-46d3-46ea-a110-f011993a0fb0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:23.551662 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4dedb-d02c-47e5-a5de-79dc3b9adc18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:23.552277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4dedb-d02c-47e5-a5de-79dc3b9adc18 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.552588 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4dedb-d02c-47e5-a5de-79dc3b9adc18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.553135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4dedb-d02c-47e5-a5de-79dc3b9adc18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.589531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1037/2062] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:23.591298 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3bddc-8e31-486e-806f-038bb1a9e177 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:23.601854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b3bddc-8e31-486e-806f-038bb1a9e177 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:23.609011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63da05-afc0-4bdf-8d9d-d262909c5be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:23.609558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63da05-afc0-4bdf-8d9d-d262909c5be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.609984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63da05-afc0-4bdf-8d9d-d262909c5be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.610299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63da05-afc0-4bdf-8d9d-d262909c5be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.610951 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63da05-afc0-4bdf-8d9d-d262909c5be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:23.612596 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63da05-afc0-4bdf-8d9d-d262909c5be0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:23.625914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1038/2063] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:23.625914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63da05-afc0-4bdf-8d9d-d262909c5be0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:23.627988 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9c63da05-afc0-4bdf-8d9d-d262909c5be0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:23.628358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4dedb-d02c-47e5-a5de-79dc3b9adc18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:23.630063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4dedb-d02c-47e5-a5de-79dc3b9adc18 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:23.642997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1026/2064] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:23.643870 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4dedb-d02c-47e5-a5de-79dc3b9adc18 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:23.658610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4dedb-d02c-47e5-a5de-79dc3b9adc18 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:23.665460 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62df87-e707-4822-86be-ac6c2922a10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:23.666213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62df87-e707-4822-86be-ac6c2922a10d admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.666554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62df87-e707-4822-86be-ac6c2922a10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.666892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62df87-e707-4822-86be-ac6c2922a10d admin admin] PATH_INFO: `/v3/projects/ac96be16ecf7477d86d75724922237f8/users/d732f56ab58848b4bfb57bf5e0f50a28/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.688776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62df87-e707-4822-86be-ac6c2922a10d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac96be16ecf7477d86d75724922237f8, user_id=d732f56ab58848b4bfb57bf5e0f50a28, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:23.693066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62df87-e707-4822-86be-ac6c2922a10d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:23.699070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63da05-afc0-4bdf-8d9d-d262909c5be0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:23.705192 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88b485-04b2-4b71-9935-decfc1a33501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:23.705787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88b485-04b2-4b71-9935-decfc1a33501 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.706128 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88b485-04b2-4b71-9935-decfc1a33501 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.706604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88b485-04b2-4b71-9935-decfc1a33501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.717714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1027/2065] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:23 2022] PUT /identity/v3/projects/ac96be16ecf7477d86d75724922237f8/users/d732f56ab58848b4bfb57bf5e0f50a28/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:23.718965 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88b485-04b2-4b71-9935-decfc1a33501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:23.719695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62df87-e707-4822-86be-ac6c2922a10d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:23.721137 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a88b485-04b2-4b71-9935-decfc1a33501 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:23.733639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1039/2066] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:23.817638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62df87-e707-4822-86be-ac6c2922a10d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:23.825620 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dd415-d54f-4e7a-95e4-bcbfc53d7767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:23.826455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dd415-d54f-4e7a-95e4-bcbfc53d7767 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.826832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dd415-d54f-4e7a-95e4-bcbfc53d7767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.827274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dd415-d54f-4e7a-95e4-bcbfc53d7767 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.828473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642dd415-d54f-4e7a-95e4-bcbfc53d7767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:23.830788 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642dd415-d54f-4e7a-95e4-bcbfc53d7767 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:23.846757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1028/2067] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:23.850814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88b485-04b2-4b71-9935-decfc1a33501 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:23.864590 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88b485-04b2-4b71-9935-decfc1a33501 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:23.872616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c2e07-4572-4c5f-b168-a07ac255a2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:23.873296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c2e07-4572-4c5f-b168-a07ac255a2f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.873593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c2e07-4572-4c5f-b168-a07ac255a2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.873837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c2e07-4572-4c5f-b168-a07ac255a2f9 admin admin] PATH_INFO: `/v3/projects/ac96be16ecf7477d86d75724922237f8/users/d732f56ab58848b4bfb57bf5e0f50a28/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.878042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c2e07-4572-4c5f-b168-a07ac255a2f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac96be16ecf7477d86d75724922237f8, user_id=d732f56ab58848b4bfb57bf5e0f50a28, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:23.881502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c2e07-4572-4c5f-b168-a07ac255a2f9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:23.904070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1040/2068] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:23 2022] PUT /identity/v3/projects/ac96be16ecf7477d86d75724922237f8/users/d732f56ab58848b4bfb57bf5e0f50a28/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:23.917378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dcf1f-42e1-42b4-b64c-2a520c163007 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:23.917683 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dcf1f-42e1-42b4-b64c-2a520c163007 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:23.918061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dcf1f-42e1-42b4-b64c-2a520c163007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:23.964157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:23.971907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d66dcf1f-42e1-42b4-b64c-2a520c163007 None None] MFA Rules not processed for user `d732f56ab58848b4bfb57bf5e0f50a28`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:23.973412 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d66dcf1f-42e1-42b4-b64c-2a520c163007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:24.120626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1029/2069] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:24.193102 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c2e07-4572-4c5f-b168-a07ac255a2f9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:24.236162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:24.275911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c2e07-4572-4c5f-b168-a07ac255a2f9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:24.280501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95297d4e-9395-4bc7-a89f-be082215f62b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:24.281448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297d4e-9395-4bc7-a89f-be082215f62b service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:24.281787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297d4e-9395-4bc7-a89f-be082215f62b service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:24.282263 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95297d4e-9395-4bc7-a89f-be082215f62b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:24.295824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95297d4e-9395-4bc7-a89f-be082215f62b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:24.297395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95297d4e-9395-4bc7-a89f-be082215f62b service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:24.340581 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1041/2070] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:24.448033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dcf1f-42e1-42b4-b64c-2a520c163007 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:24.459473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dcf1f-42e1-42b4-b64c-2a520c163007 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:24.465851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f99d33-0369-4e8e-82fd-b2c381ab0c3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:24.466593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f99d33-0369-4e8e-82fd-b2c381ab0c3d service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:24.466939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f99d33-0369-4e8e-82fd-b2c381ab0c3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:24.469130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f99d33-0369-4e8e-82fd-b2c381ab0c3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:24.538133 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f99d33-0369-4e8e-82fd-b2c381ab0c3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:24.539685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f99d33-0369-4e8e-82fd-b2c381ab0c3d service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:24.549964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1030/2071] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:24.709158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff797f5-b01a-422e-933c-0a5dc7378f88 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:24.709437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff797f5-b01a-422e-933c-0a5dc7378f88 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:24.709705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff797f5-b01a-422e-933c-0a5dc7378f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:24.748198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-cff797f5-b01a-422e-933c-0a5dc7378f88 None None] MFA Rules not processed for user `1dba953d778c418abea22efac408e2db`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:24.749499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cff797f5-b01a-422e-933c-0a5dc7378f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:24.817400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1042/2072] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:24.828194 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f99d33-0369-4e8e-82fd-b2c381ab0c3d service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:24.836536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f99d33-0369-4e8e-82fd-b2c381ab0c3d service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:24.840924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cbca8-badb-4aa2-b60b-cf4f30a22f1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:24.841524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cbca8-badb-4aa2-b60b-cf4f30a22f1d service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:24.841792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cbca8-badb-4aa2-b60b-cf4f30a22f1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:24.842172 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6cbca8-badb-4aa2-b60b-cf4f30a22f1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:24.851310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cbca8-badb-4aa2-b60b-cf4f30a22f1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:24.852762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6cbca8-badb-4aa2-b60b-cf4f30a22f1d service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:24.863423 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1031/2073] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:26.365960 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95fdd3-48cb-4904-a3e6-e6add16ad659 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:26.366681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95fdd3-48cb-4904-a3e6-e6add16ad659 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:26.366681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95fdd3-48cb-4904-a3e6-e6add16ad659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:26.388753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9d95fdd3-48cb-4904-a3e6-e6add16ad659 None None] MFA Rules not processed for user `1dba953d778c418abea22efac408e2db`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:26.390328 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9d95fdd3-48cb-4904-a3e6-e6add16ad659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:26.403897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1043/2074] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:27.119936 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cbca8-badb-4aa2-b60b-cf4f30a22f1d service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:27.121363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4a6cbca8-badb-4aa2-b60b-cf4f30a22f1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:27.197236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6cbca8-badb-4aa2-b60b-cf4f30a22f1d service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:27.201945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:27.202640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:27.202946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:27.203214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] PATH_INFO: `/v3/users/23c7f4d5aeb14dedad7f04c049dd2bff` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:27.204528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c7f4d5aeb14dedad7f04c049dd2bff)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:27.206647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:27.237143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:27.256979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c7f4d5aeb14dedad7f04c049dd2bff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:27.257886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c7f4d5aeb14dedad7f04c049dd2bff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:27.266266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c7f4d5aeb14dedad7f04c049dd2bff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:27.273616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c7f4d5aeb14dedad7f04c049dd2bff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:27.286188 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c7f4d5aeb14dedad7f04c049dd2bff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:27.292171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1032/2075] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:27 2022] DELETE /identity/v3/users/23c7f4d5aeb14dedad7f04c049dd2bff => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:25:27.296584 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181e393-58c3-44cb-9ed9-30ebaa20267b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:27.296871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181e393-58c3-44cb-9ed9-30ebaa20267b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:27.297168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181e393-58c3-44cb-9ed9-30ebaa20267b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:27.325152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b181e393-58c3-44cb-9ed9-30ebaa20267b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:27.326626 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b181e393-58c3-44cb-9ed9-30ebaa20267b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:27.415106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1044/2076] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:27.422144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:27.423833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:27.529822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057045d5-0076-4024-a739-636cc0d61478 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:27.534274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dc490-76a9-4905-b4e0-94fa2c2a7fc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:27.534944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dc490-76a9-4905-b4e0-94fa2c2a7fc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:27.535225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dc490-76a9-4905-b4e0-94fa2c2a7fc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:27.535525 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1dc490-76a9-4905-b4e0-94fa2c2a7fc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:27.547666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1dc490-76a9-4905-b4e0-94fa2c2a7fc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:27.549283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1dc490-76a9-4905-b4e0-94fa2c2a7fc7 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:27.561247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1033/2077] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:28.009089 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181e393-58c3-44cb-9ed9-30ebaa20267b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:28.010723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b181e393-58c3-44cb-9ed9-30ebaa20267b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:28.025682 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181e393-58c3-44cb-9ed9-30ebaa20267b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:28.033895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:28.035064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:28.035458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:28.036780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] PATH_INFO: `/v3/projects/78c80cdd46024f45b9d81e8ab4000a68` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:28.038472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78c80cdd46024f45b9d81e8ab4000a68)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:28.041004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:28.130912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:28.132626 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78c80cdd46024f45b9d81e8ab4000a68', 'request_id': 'req-bdd70027-27d1-4969-8b65-9045f0861703'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.144354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78c80cdd46024f45b9d81e8ab4000a68', 'request_id': 'req-bdd70027-27d1-4969-8b65-9045f0861703'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.154114 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78c80cdd46024f45b9d81e8ab4000a68', 'request_id': 'req-bdd70027-27d1-4969-8b65-9045f0861703'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.174503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] The token cache is being invalidate because project 78c80cdd46024f45b9d81e8ab4000a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:28.174997 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.178011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.187786 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1045/2078] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:28 2022] DELETE /identity/v3/projects/78c80cdd46024f45b9d81e8ab4000a68 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:28.187887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dc490-76a9-4905-b4e0-94fa2c2a7fc7 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:28.200259 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1dc490-76a9-4905-b4e0-94fa2c2a7fc7 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:28.288685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:28.289400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:28.289810 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:28.290265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] PATH_INFO: `/v3/users/1741349b41f248d6837fc7615f0f3df5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:28.292025 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1741349b41f248d6837fc7615f0f3df5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:28.295133 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:28.355981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1741349b41f248d6837fc7615f0f3df5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.356661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1741349b41f248d6837fc7615f0f3df5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.357934 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:28.367790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1741349b41f248d6837fc7615f0f3df5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.377699 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1741349b41f248d6837fc7615f0f3df5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.395105 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1741349b41f248d6837fc7615f0f3df5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.401615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1034/2079] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:28 2022] DELETE /identity/v3/users/1741349b41f248d6837fc7615f0f3df5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:28.695004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:28.696463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:28.720810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd70027-27d1-4969-8b65-9045f0861703 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:28.834542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:28.834542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:28.834542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:28.834542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] PATH_INFO: `/v3/projects/b858a38fa5594ee9a297f1f9118b8944` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:28.834934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b858a38fa5594ee9a297f1f9118b8944)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:28.834934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:28.886975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b858a38fa5594ee9a297f1f9118b8944', 'request_id': 'req-d0f2a33a-c21c-454e-9085-2606fa14a180'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.892681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b858a38fa5594ee9a297f1f9118b8944', 'request_id': 'req-d0f2a33a-c21c-454e-9085-2606fa14a180'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.899149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b858a38fa5594ee9a297f1f9118b8944', 'request_id': 'req-d0f2a33a-c21c-454e-9085-2606fa14a180'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.912155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] The token cache is being invalidate because project b858a38fa5594ee9a297f1f9118b8944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:28.912442 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.913071 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:28.914677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1046/2080] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:28 2022] DELETE /identity/v3/projects/b858a38fa5594ee9a297f1f9118b8944 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:28.919318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:28.921872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:28.935412 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d99a46-3475-47fe-84cb-dcdc546d34aa admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:28.994245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be338358-8b45-46f1-b8be-b471c4d3b161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:28.994878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be338358-8b45-46f1-b8be-b471c4d3b161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:28.995158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be338358-8b45-46f1-b8be-b471c4d3b161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:28.995432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be338358-8b45-46f1-b8be-b471c4d3b161 admin admin] PATH_INFO: `/v3/users/910b2df9a34c46a48caa8eea1e30e297` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:28.997136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be338358-8b45-46f1-b8be-b471c4d3b161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=910b2df9a34c46a48caa8eea1e30e297)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:28.999090 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be338358-8b45-46f1-b8be-b471c4d3b161 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:29.045957 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-be338358-8b45-46f1-b8be-b471c4d3b161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910b2df9a34c46a48caa8eea1e30e297'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:29.046630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-be338358-8b45-46f1-b8be-b471c4d3b161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910b2df9a34c46a48caa8eea1e30e297'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:29.053484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-be338358-8b45-46f1-b8be-b471c4d3b161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910b2df9a34c46a48caa8eea1e30e297'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:29.061796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-be338358-8b45-46f1-b8be-b471c4d3b161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910b2df9a34c46a48caa8eea1e30e297'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:29.075088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-be338358-8b45-46f1-b8be-b471c4d3b161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910b2df9a34c46a48caa8eea1e30e297'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:29.081623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1035/2081] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:28 2022] DELETE /identity/v3/users/910b2df9a34c46a48caa8eea1e30e297 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:29.372387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:29.374187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:29.395025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2a33a-c21c-454e-9085-2606fa14a180 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:29.473916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85735e-9706-4db5-be7d-ef2392876366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:29.474700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85735e-9706-4db5-be7d-ef2392876366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:29.475048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85735e-9706-4db5-be7d-ef2392876366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:29.475425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85735e-9706-4db5-be7d-ef2392876366 admin admin] PATH_INFO: `/v3/projects/c2483add22c2473098ea696c980c6010` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:29.476676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85735e-9706-4db5-be7d-ef2392876366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2483add22c2473098ea696c980c6010)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:29.478735 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85735e-9706-4db5-be7d-ef2392876366 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:29.489211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:29.547053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3c85735e-9706-4db5-be7d-ef2392876366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2483add22c2473098ea696c980c6010', 'request_id': 'req-3c85735e-9706-4db5-be7d-ef2392876366'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:29.553898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3c85735e-9706-4db5-be7d-ef2392876366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2483add22c2473098ea696c980c6010', 'request_id': 'req-3c85735e-9706-4db5-be7d-ef2392876366'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:29.560268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3c85735e-9706-4db5-be7d-ef2392876366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2483add22c2473098ea696c980c6010', 'request_id': 'req-3c85735e-9706-4db5-be7d-ef2392876366'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:29.574581 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3c85735e-9706-4db5-be7d-ef2392876366 admin admin] The token cache is being invalidate because project c2483add22c2473098ea696c980c6010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:29.574961 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3c85735e-9706-4db5-be7d-ef2392876366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:29.575660 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3c85735e-9706-4db5-be7d-ef2392876366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:29.577288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1047/2082] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:29 2022] DELETE /identity/v3/projects/c2483add22c2473098ea696c980c6010 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:29.583496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d57af-a912-4cab-856c-c818c40a42e8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:29.583938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d57af-a912-4cab-856c-c818c40a42e8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:29.584032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d57af-a912-4cab-856c-c818c40a42e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:29.606358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b32d57af-a912-4cab-856c-c818c40a42e8 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:29.607672 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b32d57af-a912-4cab-856c-c818c40a42e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:29.648084 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:29.668702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1036/2083] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:29.676980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e02aae-fa27-4258-8c72-0cb66bf28261 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:29.677261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e02aae-fa27-4258-8c72-0cb66bf28261 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:29.677533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e02aae-fa27-4258-8c72-0cb66bf28261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:29.696599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b0e02aae-fa27-4258-8c72-0cb66bf28261 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:29.697786 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b0e02aae-fa27-4258-8c72-0cb66bf28261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:29.706412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1048/2084] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:29.710485 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d57af-a912-4cab-856c-c818c40a42e8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:29.721911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d57af-a912-4cab-856c-c818c40a42e8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:29.727419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdab12-6185-4729-935a-945c457fdeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:29.728445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdab12-6185-4729-935a-945c457fdeaf admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:29.728958 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdab12-6185-4729-935a-945c457fdeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:29.729458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbdab12-6185-4729-935a-945c457fdeaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:29.730656 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdab12-6185-4729-935a-945c457fdeaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:25:29.730962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdab12-6185-4729-935a-945c457fdeaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:29.734061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbdab12-6185-4729-935a-945c457fdeaf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:29.746870 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1037/2085] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:25:29.751055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e02aae-fa27-4258-8c72-0cb66bf28261 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:29.760255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e02aae-fa27-4258-8c72-0cb66bf28261 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:29.765415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeda1af-ec2c-470f-a3df-161008238cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:29.766033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeda1af-ec2c-470f-a3df-161008238cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:29.766334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeda1af-ec2c-470f-a3df-161008238cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:29.766604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeda1af-ec2c-470f-a3df-161008238cd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:29.767292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeda1af-ec2c-470f-a3df-161008238cd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:29.769212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeda1af-ec2c-470f-a3df-161008238cd0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:29.797248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1049/2086] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:29 2022] POST /identity/v3/projects => generated 401 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:29.801435 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdab12-6185-4729-935a-945c457fdeaf admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:29.811386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbdab12-6185-4729-935a-945c457fdeaf admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:29.877219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f192d-938f-4b0f-ad6d-a75d5306edb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:29.877822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f192d-938f-4b0f-ad6d-a75d5306edb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:29.878129 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f192d-938f-4b0f-ad6d-a75d5306edb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:29.878405 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f192d-938f-4b0f-ad6d-a75d5306edb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:29.879037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f192d-938f-4b0f-ad6d-a75d5306edb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:29.880969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f192d-938f-4b0f-ad6d-a75d5306edb9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:29.906236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1038/2087] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:29 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:29.907700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeda1af-ec2c-470f-a3df-161008238cd0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:29.917823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeda1af-ec2c-470f-a3df-161008238cd0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:29.922769 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7afd56-246a-4bd0-86e4-98f30ed6a5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:29.923372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7afd56-246a-4bd0-86e4-98f30ed6a5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:29.923646 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7afd56-246a-4bd0-86e4-98f30ed6a5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:29.923909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7afd56-246a-4bd0-86e4-98f30ed6a5f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:29.924551 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7afd56-246a-4bd0-86e4-98f30ed6a5f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:29.926152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7afd56-246a-4bd0-86e4-98f30ed6a5f8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:29.940240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1050/2088] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:29.942009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f192d-938f-4b0f-ad6d-a75d5306edb9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:29.953135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f192d-938f-4b0f-ad6d-a75d5306edb9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:29.958195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7c353-238b-4358-ae39-474a7b3a498c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:29.958783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7c353-238b-4358-ae39-474a7b3a498c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:29.959061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7c353-238b-4358-ae39-474a7b3a498c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:29.959332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7c353-238b-4358-ae39-474a7b3a498c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:29.959983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7c353-238b-4358-ae39-474a7b3a498c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:29.961567 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7c353-238b-4358-ae39-474a7b3a498c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:29.972272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1039/2089] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:29.976584 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7afd56-246a-4bd0-86e4-98f30ed6a5f8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:29.987193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7afd56-246a-4bd0-86e4-98f30ed6a5f8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:29.991904 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6f7bd-336c-4e73-9034-d369d9d9b80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:29.992551 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6f7bd-336c-4e73-9034-d369d9d9b80c admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:29.992831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6f7bd-336c-4e73-9034-d369d9d9b80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:29.993104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6f7bd-336c-4e73-9034-d369d9d9b80c admin admin] PATH_INFO: `/v3/projects/77bd383a68b74d28a9ab8d0058268ed2/users/651ba28a91c149e78352953db5e573e5/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:30.008280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6f7bd-336c-4e73-9034-d369d9d9b80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=77bd383a68b74d28a9ab8d0058268ed2, user_id=651ba28a91c149e78352953db5e573e5, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:30.010921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6f7bd-336c-4e73-9034-d369d9d9b80c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:30.025075 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1051/2090] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:29 2022] PUT /identity/v3/projects/77bd383a68b74d28a9ab8d0058268ed2/users/651ba28a91c149e78352953db5e573e5/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:30.038031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1afbc-4f86-4047-ace5-4670ebbb05db None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:30.038344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1afbc-4f86-4047-ace5-4670ebbb05db None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:30.038620 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1afbc-4f86-4047-ace5-4670ebbb05db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:30.067966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a5e1afbc-4f86-4047-ace5-4670ebbb05db None None] MFA Rules not processed for user `651ba28a91c149e78352953db5e573e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:30.069173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a5e1afbc-4f86-4047-ace5-4670ebbb05db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:30.140458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1040/2091] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:30 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:30.145171 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6f7bd-336c-4e73-9034-d369d9d9b80c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:30.208604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6f7bd-336c-4e73-9034-d369d9d9b80c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:30.213399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d9fa3-8245-4a19-a128-c6ccc5b62fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:30.214002 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d9fa3-8245-4a19-a128-c6ccc5b62fad admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:30.214308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d9fa3-8245-4a19-a128-c6ccc5b62fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:30.214573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d9fa3-8245-4a19-a128-c6ccc5b62fad admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:30.215263 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d9fa3-8245-4a19-a128-c6ccc5b62fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:30.217155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d9fa3-8245-4a19-a128-c6ccc5b62fad admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:30.243159 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1052/2092] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:30 2022] POST /identity/v3/projects => generated 399 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:30.248019 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1afbc-4f86-4047-ace5-4670ebbb05db None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:30.258658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1afbc-4f86-4047-ace5-4670ebbb05db None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:30.295263 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:30.315528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a701a28-2a00-4f55-8e17-64899d9149fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:30.316125 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a701a28-2a00-4f55-8e17-64899d9149fd admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:30.316398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a701a28-2a00-4f55-8e17-64899d9149fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:30.316671 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a701a28-2a00-4f55-8e17-64899d9149fd admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:30.317288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a701a28-2a00-4f55-8e17-64899d9149fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:30.319211 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a701a28-2a00-4f55-8e17-64899d9149fd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:30.343627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1041/2093] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:30 2022] POST /identity/v3/users => generated 306 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:30.344807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d9fa3-8245-4a19-a128-c6ccc5b62fad admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:30.355737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d9fa3-8245-4a19-a128-c6ccc5b62fad admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:30.360460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79351b8-cf6c-491a-9e6c-e94356ef8e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:30.361054 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79351b8-cf6c-491a-9e6c-e94356ef8e70 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:30.361326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79351b8-cf6c-491a-9e6c-e94356ef8e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:30.361595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79351b8-cf6c-491a-9e6c-e94356ef8e70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:30.362299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79351b8-cf6c-491a-9e6c-e94356ef8e70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:30.363884 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79351b8-cf6c-491a-9e6c-e94356ef8e70 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:30.375224 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1053/2094] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:30.379345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a701a28-2a00-4f55-8e17-64899d9149fd admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:30.389162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a701a28-2a00-4f55-8e17-64899d9149fd admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:30.393756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c532cd2-75b7-4840-99c5-d0bfbfee39e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:30.394419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532cd2-75b7-4840-99c5-d0bfbfee39e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:30.394696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532cd2-75b7-4840-99c5-d0bfbfee39e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:30.394965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c532cd2-75b7-4840-99c5-d0bfbfee39e0 admin admin] PATH_INFO: `/v3/projects/91ad9bd3e3cd404eaaa2c19f829eb27a/users/f1824c799a4041419b79251aabe01c2c/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:30.422128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c532cd2-75b7-4840-99c5-d0bfbfee39e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ad9bd3e3cd404eaaa2c19f829eb27a, user_id=f1824c799a4041419b79251aabe01c2c, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:30.422861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c532cd2-75b7-4840-99c5-d0bfbfee39e0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:30.438018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1042/2095] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:30 2022] PUT /identity/v3/projects/91ad9bd3e3cd404eaaa2c19f829eb27a/users/f1824c799a4041419b79251aabe01c2c/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:30.442598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79351b8-cf6c-491a-9e6c-e94356ef8e70 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:30.452763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79351b8-cf6c-491a-9e6c-e94356ef8e70 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:30.457648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:30.520459 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b76c7-6dc4-45b5-8c60-000609f9e729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:30.520980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b76c7-6dc4-45b5-8c60-000609f9e729 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:30.521258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b76c7-6dc4-45b5-8c60-000609f9e729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:30.521526 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b76c7-6dc4-45b5-8c60-000609f9e729 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:30.522189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b76c7-6dc4-45b5-8c60-000609f9e729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:30.523776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b76c7-6dc4-45b5-8c60-000609f9e729 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:30.533355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1054/2096] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:30.537648 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c532cd2-75b7-4840-99c5-d0bfbfee39e0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:30.548664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c532cd2-75b7-4840-99c5-d0bfbfee39e0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:30.553276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40864398-089d-43cb-bfa5-50f603a36063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:30.553915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40864398-089d-43cb-bfa5-50f603a36063 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:30.554228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40864398-089d-43cb-bfa5-50f603a36063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:30.554505 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40864398-089d-43cb-bfa5-50f603a36063 admin admin] PATH_INFO: `/v3/projects/91ad9bd3e3cd404eaaa2c19f829eb27a/users/f1824c799a4041419b79251aabe01c2c/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:30.556363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40864398-089d-43cb-bfa5-50f603a36063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ad9bd3e3cd404eaaa2c19f829eb27a, user_id=f1824c799a4041419b79251aabe01c2c, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:30.559657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40864398-089d-43cb-bfa5-50f603a36063 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:30.569634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1043/2097] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:30 2022] PUT /identity/v3/projects/91ad9bd3e3cd404eaaa2c19f829eb27a/users/f1824c799a4041419b79251aabe01c2c/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:30.579670 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94573f83-a721-4062-a94c-7d623a9efbaf None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:30.579954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94573f83-a721-4062-a94c-7d623a9efbaf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:30.580224 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94573f83-a721-4062-a94c-7d623a9efbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:30.607378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-94573f83-a721-4062-a94c-7d623a9efbaf None None] MFA Rules not processed for user `f1824c799a4041419b79251aabe01c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:30.608578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-94573f83-a721-4062-a94c-7d623a9efbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:30.677790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1055/2098] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:30 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:30.688923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40864398-089d-43cb-bfa5-50f603a36063 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:30.690718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-40864398-089d-43cb-bfa5-50f603a36063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:30.758558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40864398-089d-43cb-bfa5-50f603a36063 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:30.762257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cde5d-9c86-470e-b243-3e8cb225e4d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:30.762906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cde5d-9c86-470e-b243-3e8cb225e4d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:30.763181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cde5d-9c86-470e-b243-3e8cb225e4d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:30.763455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467cde5d-9c86-470e-b243-3e8cb225e4d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:30.777640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467cde5d-9c86-470e-b243-3e8cb225e4d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:30.779069 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467cde5d-9c86-470e-b243-3e8cb225e4d1 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:30.788446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1044/2099] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:30 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:30.876395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94573f83-a721-4062-a94c-7d623a9efbaf None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:30.885753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94573f83-a721-4062-a94c-7d623a9efbaf None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:30.889324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376af745-4a69-4d2c-8bda-e33d81e82c09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:30.889910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376af745-4a69-4d2c-8bda-e33d81e82c09 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:30.890224 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376af745-4a69-4d2c-8bda-e33d81e82c09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:30.890496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376af745-4a69-4d2c-8bda-e33d81e82c09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:30.945906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376af745-4a69-4d2c-8bda-e33d81e82c09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:30.947354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376af745-4a69-4d2c-8bda-e33d81e82c09 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:30.956863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1056/2100] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:30 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:37.850939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cde5d-9c86-470e-b243-3e8cb225e4d1 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:37.852477 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-467cde5d-9c86-470e-b243-3e8cb225e4d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:37.925865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467cde5d-9c86-470e-b243-3e8cb225e4d1 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:37.931230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6094c2-2ed2-4163-b834-3f4f0d3c6ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:37.932049 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6094c2-2ed2-4163-b834-3f4f0d3c6ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:37.932360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6094c2-2ed2-4163-b834-3f4f0d3c6ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:37.932659 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6094c2-2ed2-4163-b834-3f4f0d3c6ec1 admin admin] PATH_INFO: `/v3/users/961ce0af5d934b45999a0630b5101fa2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:37.933915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6094c2-2ed2-4163-b834-3f4f0d3c6ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=961ce0af5d934b45999a0630b5101fa2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:37.937606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6094c2-2ed2-4163-b834-3f4f0d3c6ec1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:37.986573 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:38.003781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6c6094c2-2ed2-4163-b834-3f4f0d3c6ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '961ce0af5d934b45999a0630b5101fa2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.006440 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6c6094c2-2ed2-4163-b834-3f4f0d3c6ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '961ce0af5d934b45999a0630b5101fa2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.017801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376af745-4a69-4d2c-8bda-e33d81e82c09 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:38.019324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-376af745-4a69-4d2c-8bda-e33d81e82c09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:38.023880 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6c6094c2-2ed2-4163-b834-3f4f0d3c6ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '961ce0af5d934b45999a0630b5101fa2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.038457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376af745-4a69-4d2c-8bda-e33d81e82c09 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:38.051183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6c6094c2-2ed2-4163-b834-3f4f0d3c6ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '961ce0af5d934b45999a0630b5101fa2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.066165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d41aba-8959-4803-a6b2-d8981deffdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:38.066827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d41aba-8959-4803-a6b2-d8981deffdaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:38.067178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d41aba-8959-4803-a6b2-d8981deffdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:38.067360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d41aba-8959-4803-a6b2-d8981deffdaf admin admin] PATH_INFO: `/v3/users/611ff79435f04fa198640f599a254076` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:38.068710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6c6094c2-2ed2-4163-b834-3f4f0d3c6ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '961ce0af5d934b45999a0630b5101fa2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.070591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d41aba-8959-4803-a6b2-d8981deffdaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=611ff79435f04fa198640f599a254076)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:38.072738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d41aba-8959-4803-a6b2-d8981deffdaf admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:38.090687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1045/2101] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:37 2022] DELETE /identity/v3/users/961ce0af5d934b45999a0630b5101fa2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:38.095735 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564800c-bf82-4186-90fb-ea5ce47692ca None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:38.096198 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564800c-bf82-4186-90fb-ea5ce47692ca None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:38.096644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564800c-bf82-4186-90fb-ea5ce47692ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:38.101021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:38.123376 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5564800c-bf82-4186-90fb-ea5ce47692ca None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:38.124989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5564800c-bf82-4186-90fb-ea5ce47692ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:38.135501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c3d41aba-8959-4803-a6b2-d8981deffdaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '611ff79435f04fa198640f599a254076'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.144973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c3d41aba-8959-4803-a6b2-d8981deffdaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '611ff79435f04fa198640f599a254076'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.145978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c3d41aba-8959-4803-a6b2-d8981deffdaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '611ff79435f04fa198640f599a254076'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.156821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c3d41aba-8959-4803-a6b2-d8981deffdaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '611ff79435f04fa198640f599a254076'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.180759 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c3d41aba-8959-4803-a6b2-d8981deffdaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '611ff79435f04fa198640f599a254076'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.187637 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1057/2102] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:38 2022] DELETE /identity/v3/users/611ff79435f04fa198640f599a254076 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:38.192442 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bc4f6-eb75-4966-bee8-1f5495c7042d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:38.193499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bc4f6-eb75-4966-bee8-1f5495c7042d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:38.193499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bc4f6-eb75-4966-bee8-1f5495c7042d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:38.216853 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1046/2103] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:38.222095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f43bc4f6-eb75-4966-bee8-1f5495c7042d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:38.223532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f43bc4f6-eb75-4966-bee8-1f5495c7042d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:38.227081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564800c-bf82-4186-90fb-ea5ce47692ca None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:38.228846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5564800c-bf82-4186-90fb-ea5ce47692ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:38.305198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1058/2104] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:38.312373 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564800c-bf82-4186-90fb-ea5ce47692ca None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:38.323340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bc4f6-eb75-4966-bee8-1f5495c7042d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:38.376152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bbb78-b168-4344-870d-ab70bd09dfe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:38.376753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bbb78-b168-4344-870d-ab70bd09dfe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:38.377031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bbb78-b168-4344-870d-ab70bd09dfe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:38.377300 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bbb78-b168-4344-870d-ab70bd09dfe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:38.387185 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bbb78-b168-4344-870d-ab70bd09dfe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:38.389348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1bbb78-b168-4344-870d-ab70bd09dfe5 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:38.390160 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bc4f6-eb75-4966-bee8-1f5495c7042d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:38.393958 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a0148-3e1e-4400-82dd-3d6adbe8c37f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:38.394682 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a0148-3e1e-4400-82dd-3d6adbe8c37f service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:38.395063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a0148-3e1e-4400-82dd-3d6adbe8c37f service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:38.395412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a0148-3e1e-4400-82dd-3d6adbe8c37f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:38.400439 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1047/2105] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:25:38.408665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a0148-3e1e-4400-82dd-3d6adbe8c37f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:38.410198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a0148-3e1e-4400-82dd-3d6adbe8c37f service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:38.422186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1059/2106] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:25:38.864999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bbb78-b168-4344-870d-ab70bd09dfe5 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:38.866703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4c1bbb78-b168-4344-870d-ab70bd09dfe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:38.880967 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a0148-3e1e-4400-82dd-3d6adbe8c37f service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:38.882559 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-765a0148-3e1e-4400-82dd-3d6adbe8c37f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:38.891625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bbb78-b168-4344-870d-ab70bd09dfe5 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:38.896643 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c258c-52db-492c-8ae9-252aa3ac16d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:38.897552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c258c-52db-492c-8ae9-252aa3ac16d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:38.898351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c258c-52db-492c-8ae9-252aa3ac16d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:38.898351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c258c-52db-492c-8ae9-252aa3ac16d0 admin admin] PATH_INFO: `/v3/projects/7d31e5cd47024ab49127c5ee7f0bbc87` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:38.899722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c258c-52db-492c-8ae9-252aa3ac16d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d31e5cd47024ab49127c5ee7f0bbc87)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:38.902181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c258c-52db-492c-8ae9-252aa3ac16d0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:38.903913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a0148-3e1e-4400-82dd-3d6adbe8c37f service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:38.912016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:38.918265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:38.919862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:38.920143 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:38.920413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b admin admin] PATH_INFO: `/v3/projects/f20f02f037ee463f9cd93ad3eece56ba` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:38.921703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20f02f037ee463f9cd93ad3eece56ba)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:38.923798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:38.966406 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ce5c258c-52db-492c-8ae9-252aa3ac16d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d31e5cd47024ab49127c5ee7f0bbc87', 'request_id': 'req-ce5c258c-52db-492c-8ae9-252aa3ac16d0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.968589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:38.973579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ce5c258c-52db-492c-8ae9-252aa3ac16d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d31e5cd47024ab49127c5ee7f0bbc87', 'request_id': 'req-ce5c258c-52db-492c-8ae9-252aa3ac16d0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.979369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20f02f037ee463f9cd93ad3eece56ba', 'request_id': 'req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.981409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ce5c258c-52db-492c-8ae9-252aa3ac16d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d31e5cd47024ab49127c5ee7f0bbc87', 'request_id': 'req-ce5c258c-52db-492c-8ae9-252aa3ac16d0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.987823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20f02f037ee463f9cd93ad3eece56ba', 'request_id': 'req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.994776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20f02f037ee463f9cd93ad3eece56ba', 'request_id': 'req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.995275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ce5c258c-52db-492c-8ae9-252aa3ac16d0 admin admin] The token cache is being invalidate because project 7d31e5cd47024ab49127c5ee7f0bbc87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:38.995565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ce5c258c-52db-492c-8ae9-252aa3ac16d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.996275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ce5c258c-52db-492c-8ae9-252aa3ac16d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:38.998047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1048/2107] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:38 2022] DELETE /identity/v3/projects/7d31e5cd47024ab49127c5ee7f0bbc87 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:39.004236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990a32c-92d0-4826-9a1b-2743aa7d6752 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.004512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990a32c-92d0-4826-9a1b-2743aa7d6752 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.004785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990a32c-92d0-4826-9a1b-2743aa7d6752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.008390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b admin admin] The token cache is being invalidate because project f20f02f037ee463f9cd93ad3eece56ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:39.008676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:39.009287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ece628c1-e5b7-4173-aa9c-0dcd76484e7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:39.010783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1060/2108] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:38 2022] DELETE /identity/v3/projects/f20f02f037ee463f9cd93ad3eece56ba => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:25:39.015372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d20e0-92d7-4239-bb15-f6335c932f71 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.015657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d20e0-92d7-4239-bb15-f6335c932f71 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.016072 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d20e0-92d7-4239-bb15-f6335c932f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.030884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0990a32c-92d0-4826-9a1b-2743aa7d6752 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:39.032186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0990a32c-92d0-4826-9a1b-2743aa7d6752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:39.039268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ba6d20e0-92d7-4239-bb15-f6335c932f71 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:39.040521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ba6d20e0-92d7-4239-bb15-f6335c932f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:39.098800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1049/2109] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:39.107382 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22922de2-cf54-455f-851d-0ecdd895aa9c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.107714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22922de2-cf54-455f-851d-0ecdd895aa9c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.107915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22922de2-cf54-455f-851d-0ecdd895aa9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.110400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1061/2110] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:39.119221 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628001a-2ad7-4d4a-8717-53f6ba44c2f8 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.119506 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628001a-2ad7-4d4a-8717-53f6ba44c2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.119794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628001a-2ad7-4d4a-8717-53f6ba44c2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.130544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-22922de2-cf54-455f-851d-0ecdd895aa9c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:39.131742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-22922de2-cf54-455f-851d-0ecdd895aa9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:39.141712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a628001a-2ad7-4d4a-8717-53f6ba44c2f8 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:39.142967 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a628001a-2ad7-4d4a-8717-53f6ba44c2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:39.155381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1062/2111] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:39.155484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1050/2112] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:39.159017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628001a-2ad7-4d4a-8717-53f6ba44c2f8 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.160313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22922de2-cf54-455f-851d-0ecdd895aa9c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.172104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628001a-2ad7-4d4a-8717-53f6ba44c2f8 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.173556 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22922de2-cf54-455f-851d-0ecdd895aa9c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.177578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cdc9b-4d54-4f69-835b-24de575b1545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.178220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cdc9b-4d54-4f69-835b-24de575b1545 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.178623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cdc9b-4d54-4f69-835b-24de575b1545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.178898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cdc9b-4d54-4f69-835b-24de575b1545 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.179543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cdc9b-4d54-4f69-835b-24de575b1545 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:25:39.179865 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cdc9b-4d54-4f69-835b-24de575b1545 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.180046 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad2b62-60db-4687-9651-a7ca9adcfd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.181380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad2b62-60db-4687-9651-a7ca9adcfd76 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.181476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cdc9b-4d54-4f69-835b-24de575b1545 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.182063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad2b62-60db-4687-9651-a7ca9adcfd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.182434 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad2b62-60db-4687-9651-a7ca9adcfd76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.183285 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad2b62-60db-4687-9651-a7ca9adcfd76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:25:39.183652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad2b62-60db-4687-9651-a7ca9adcfd76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.185469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad2b62-60db-4687-9651-a7ca9adcfd76 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.195474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1063/2113] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:25:39.201119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cdc9b-4d54-4f69-835b-24de575b1545 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.210116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1051/2114] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:25:39.215094 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad2b62-60db-4687-9651-a7ca9adcfd76 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.217160 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cdc9b-4d54-4f69-835b-24de575b1545 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.223708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590165e-f8e7-4487-8d89-08fed53a7051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.224492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590165e-f8e7-4487-8d89-08fed53a7051 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.224911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590165e-f8e7-4487-8d89-08fed53a7051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.225440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590165e-f8e7-4487-8d89-08fed53a7051 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.226832 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590165e-f8e7-4487-8d89-08fed53a7051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.227837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad2b62-60db-4687-9651-a7ca9adcfd76 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.229669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590165e-f8e7-4487-8d89-08fed53a7051 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.233536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa1c20-1058-4b50-a7b0-67cd3562e3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.234013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa1c20-1058-4b50-a7b0-67cd3562e3bb admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.234484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa1c20-1058-4b50-a7b0-67cd3562e3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.234847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa1c20-1058-4b50-a7b0-67cd3562e3bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.235695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa1c20-1058-4b50-a7b0-67cd3562e3bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.237838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa1c20-1058-4b50-a7b0-67cd3562e3bb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.265017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1064/2115] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:39 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:39.269878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590165e-f8e7-4487-8d89-08fed53a7051 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.276352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1052/2116] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:39 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:39.277137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa1c20-1058-4b50-a7b0-67cd3562e3bb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.354928 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:39.355893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590165e-f8e7-4487-8d89-08fed53a7051 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.361180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa1c20-1058-4b50-a7b0-67cd3562e3bb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.364615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b07187-07a0-4d78-b143-c6a4a470529b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.365591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b07187-07a0-4d78-b143-c6a4a470529b admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.366095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b07187-07a0-4d78-b143-c6a4a470529b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.366543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b07187-07a0-4d78-b143-c6a4a470529b admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.367243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b07187-07a0-4d78-b143-c6a4a470529b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.369498 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b07187-07a0-4d78-b143-c6a4a470529b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.374609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a6ec0-41fd-4b68-9261-40d2800bd8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.375579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a6ec0-41fd-4b68-9261-40d2800bd8d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.375965 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a6ec0-41fd-4b68-9261-40d2800bd8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.376245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a6ec0-41fd-4b68-9261-40d2800bd8d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.376959 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a6ec0-41fd-4b68-9261-40d2800bd8d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.379616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a6ec0-41fd-4b68-9261-40d2800bd8d0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.395387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1065/2117] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:39 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:25:39.400241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b07187-07a0-4d78-b143-c6a4a470529b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.411454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1053/2118] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:39 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:39.412996 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b07187-07a0-4d78-b143-c6a4a470529b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.415723 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a6ec0-41fd-4b68-9261-40d2800bd8d0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.419883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba4c28-708e-4869-a669-edf776ebb40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.420557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba4c28-708e-4869-a669-edf776ebb40f admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.420904 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba4c28-708e-4869-a669-edf776ebb40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.421238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba4c28-708e-4869-a669-edf776ebb40f admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.422152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba4c28-708e-4869-a669-edf776ebb40f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.423989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba4c28-708e-4869-a669-edf776ebb40f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.429252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a6ec0-41fd-4b68-9261-40d2800bd8d0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.436973 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77f5b5-75e7-48ef-a13e-dce1388f8e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.440115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1066/2119] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:39.440204 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77f5b5-75e7-48ef-a13e-dce1388f8e81 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.440204 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77f5b5-75e7-48ef-a13e-dce1388f8e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.440204 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77f5b5-75e7-48ef-a13e-dce1388f8e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.440808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba4c28-708e-4869-a669-edf776ebb40f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.441033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77f5b5-75e7-48ef-a13e-dce1388f8e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.442741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77f5b5-75e7-48ef-a13e-dce1388f8e81 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.446857 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:39.453346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba4c28-708e-4869-a669-edf776ebb40f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.457508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1054/2120] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:39.461894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77f5b5-75e7-48ef-a13e-dce1388f8e81 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.475492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba5e7d-bc9b-44f8-be08-d2eb52bed6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.476659 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77f5b5-75e7-48ef-a13e-dce1388f8e81 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.476819 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba5e7d-bc9b-44f8-be08-d2eb52bed6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.477086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba5e7d-bc9b-44f8-be08-d2eb52bed6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.477467 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ba5e7d-bc9b-44f8-be08-d2eb52bed6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.478456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba5e7d-bc9b-44f8-be08-d2eb52bed6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.480902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ba5e7d-bc9b-44f8-be08-d2eb52bed6d9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.490675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b2d67-faba-44e6-963b-0051e2d0e472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.491199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b2d67-faba-44e6-963b-0051e2d0e472 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.491600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b2d67-faba-44e6-963b-0051e2d0e472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.491854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b2d67-faba-44e6-963b-0051e2d0e472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.492496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b2d67-faba-44e6-963b-0051e2d0e472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.494584 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b2d67-faba-44e6-963b-0051e2d0e472 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.499989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1067/2121] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:25:39.501435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba5e7d-bc9b-44f8-be08-d2eb52bed6d9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.513164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ba5e7d-bc9b-44f8-be08-d2eb52bed6d9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.513380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1055/2122] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:39.517538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b2d67-faba-44e6-963b-0051e2d0e472 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.519895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d9757-3b8a-44fc-b204-b8e160a69af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.520938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d9757-3b8a-44fc-b204-b8e160a69af7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.521221 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d9757-3b8a-44fc-b204-b8e160a69af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.521561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d9757-3b8a-44fc-b204-b8e160a69af7 admin admin] PATH_INFO: `/v3/projects/61dd2124e7264f8f9cb4d52c5ef2dec1/users/a4efbc38f37340f3bec57265d7daf9d2/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.529006 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b2d67-faba-44e6-963b-0051e2d0e472 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.538538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149aa49-6ee4-4366-b9af-35dfdfcd72e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.539263 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149aa49-6ee4-4366-b9af-35dfdfcd72e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.539601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149aa49-6ee4-4366-b9af-35dfdfcd72e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.539876 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149aa49-6ee4-4366-b9af-35dfdfcd72e5 admin admin] PATH_INFO: `/v3/projects/6675c176d490466bac2697801fcfe5e8/users/8209748fe1f344caa843b316ad2db2d7/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.542452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d9757-3b8a-44fc-b204-b8e160a69af7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61dd2124e7264f8f9cb4d52c5ef2dec1, user_id=a4efbc38f37340f3bec57265d7daf9d2, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.546951 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d9757-3b8a-44fc-b204-b8e160a69af7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.563185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1068/2123] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:39 2022] PUT /identity/v3/projects/61dd2124e7264f8f9cb4d52c5ef2dec1/users/a4efbc38f37340f3bec57265d7daf9d2/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:25:39.566469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149aa49-6ee4-4366-b9af-35dfdfcd72e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6675c176d490466bac2697801fcfe5e8, user_id=8209748fe1f344caa843b316ad2db2d7, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.569151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149aa49-6ee4-4366-b9af-35dfdfcd72e5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.576025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca0b02-2f16-449b-9db2-1f75734f23fb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.576618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca0b02-2f16-449b-9db2-1f75734f23fb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.576947 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca0b02-2f16-449b-9db2-1f75734f23fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.583837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1056/2124] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:39 2022] PUT /identity/v3/projects/6675c176d490466bac2697801fcfe5e8/users/8209748fe1f344caa843b316ad2db2d7/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:25:39.595769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcbf89-cf1a-43a0-8c47-ee42370391ac None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.596076 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcbf89-cf1a-43a0-8c47-ee42370391ac None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.596432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcbf89-cf1a-43a0-8c47-ee42370391ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.615738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0bca0b02-2f16-449b-9db2-1f75734f23fb None None] MFA Rules not processed for user `a4efbc38f37340f3bec57265d7daf9d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:39.616987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0bca0b02-2f16-449b-9db2-1f75734f23fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:39.630579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c2fcbf89-cf1a-43a0-8c47-ee42370391ac None None] MFA Rules not processed for user `8209748fe1f344caa843b316ad2db2d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:39.631789 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c2fcbf89-cf1a-43a0-8c47-ee42370391ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:39.696236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1069/2125] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:39.710755 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca0b02-2f16-449b-9db2-1f75734f23fb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.712414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0bca0b02-2f16-449b-9db2-1f75734f23fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:39.715054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1057/2126] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:39.726658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fcbf89-cf1a-43a0-8c47-ee42370391ac None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:39.728042 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c2fcbf89-cf1a-43a0-8c47-ee42370391ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:39.791752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca0b02-2f16-449b-9db2-1f75734f23fb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.796269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc5941-c8d7-43ef-b461-661867c58ff8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc89c1a7e684612a73ec9fef6bda780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.796918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc5941-c8d7-43ef-b461-661867c58ff8 service glance] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.797235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc5941-c8d7-43ef-b461-661867c58ff8 service glance] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.797530 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc5941-c8d7-43ef-b461-661867c58ff8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.797630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fcbf89-cf1a-43a0-8c47-ee42370391ac None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:39.801427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719c80f-a607-4f40-99e5-0893357678ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:39.802023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719c80f-a607-4f40-99e5-0893357678ce service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:39.802418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719c80f-a607-4f40-99e5-0893357678ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:39.802596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719c80f-a607-4f40-99e5-0893357678ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:39.807349 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc5941-c8d7-43ef-b461-661867c58ff8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.808845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc5941-c8d7-43ef-b461-661867c58ff8 service glance] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.812945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719c80f-a607-4f40-99e5-0893357678ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:39.814395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719c80f-a607-4f40-99e5-0893357678ce service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:39.818946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1070/2127] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:39.825052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1058/2128] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:25:40.724783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc5941-c8d7-43ef-b461-661867c58ff8 service glance] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:40.726233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-33dc5941-c8d7-43ef-b461-661867c58ff8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:40.802744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc5941-c8d7-43ef-b461-661867c58ff8 service glance] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:40.810467 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:40.811213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:40.811560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:40.812051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] PATH_INFO: `/v3/users/651ba28a91c149e78352953db5e573e5` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:40.813130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] RBAC: Authorizing `identity:delete_user(user_id=651ba28a91c149e78352953db5e573e5)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:40.815269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:40.817290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:40.894360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '651ba28a91c149e78352953db5e573e5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:40.901782 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '651ba28a91c149e78352953db5e573e5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:40.902531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '651ba28a91c149e78352953db5e573e5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:40.911394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '651ba28a91c149e78352953db5e573e5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:40.928026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '651ba28a91c149e78352953db5e573e5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:40.935333 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1071/2129] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:40 2022] DELETE /identity/v3/users/651ba28a91c149e78352953db5e573e5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:40.940659 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680a148-95da-443e-98c4-b4d19ca18ae3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:40.941187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680a148-95da-443e-98c4-b4d19ca18ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:40.941592 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680a148-95da-443e-98c4-b4d19ca18ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:40.964385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c680a148-95da-443e-98c4-b4d19ca18ae3 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:40.965705 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c680a148-95da-443e-98c4-b4d19ca18ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:41.000280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:41.050899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1059/2130] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:41.061721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:41.063202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:41.135296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4d440-5ab0-451e-8b05-fbb1a7cf850a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:41.139226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b162d01-81ef-41b7-b323-3d2122c4f73a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:41.139927 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b162d01-81ef-41b7-b323-3d2122c4f73a service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:41.140241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b162d01-81ef-41b7-b323-3d2122c4f73a service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:41.140572 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b162d01-81ef-41b7-b323-3d2122c4f73a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:41.156775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b162d01-81ef-41b7-b323-3d2122c4f73a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:41.160864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b162d01-81ef-41b7-b323-3d2122c4f73a service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:41.185351 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1072/2131] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:41.650966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680a148-95da-443e-98c4-b4d19ca18ae3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:41.652497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c680a148-95da-443e-98c4-b4d19ca18ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:41.666560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680a148-95da-443e-98c4-b4d19ca18ae3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:41.671486 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:41.672334 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:41.672438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:41.672805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] PATH_INFO: `/v3/projects/77bd383a68b74d28a9ab8d0058268ed2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:41.674512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=77bd383a68b74d28a9ab8d0058268ed2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:41.676393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:41.731170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77bd383a68b74d28a9ab8d0058268ed2', 'request_id': 'req-ae9c2ccc-100e-4cd0-9e31-358328799bde'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:41.737141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77bd383a68b74d28a9ab8d0058268ed2', 'request_id': 'req-ae9c2ccc-100e-4cd0-9e31-358328799bde'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:41.743955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77bd383a68b74d28a9ab8d0058268ed2', 'request_id': 'req-ae9c2ccc-100e-4cd0-9e31-358328799bde'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:41.760066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] The token cache is being invalidate because project 77bd383a68b74d28a9ab8d0058268ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:41.760440 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:41.761141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:41.763313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1060/2132] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:41 2022] DELETE /identity/v3/projects/77bd383a68b74d28a9ab8d0058268ed2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:41.767555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b162d01-81ef-41b7-b323-3d2122c4f73a service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:41.778979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b162d01-81ef-41b7-b323-3d2122c4f73a service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:41.839795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3a70f-7285-4b63-97f7-0b8f6370ba30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:41.840334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3a70f-7285-4b63-97f7-0b8f6370ba30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:41.841100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3a70f-7285-4b63-97f7-0b8f6370ba30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:41.841365 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3a70f-7285-4b63-97f7-0b8f6370ba30 admin admin] PATH_INFO: `/v3/users/f1824c799a4041419b79251aabe01c2c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:41.842548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f3a70f-7285-4b63-97f7-0b8f6370ba30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1824c799a4041419b79251aabe01c2c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:41.844456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f3a70f-7285-4b63-97f7-0b8f6370ba30 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:41.867792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:41.891145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a4f3a70f-7285-4b63-97f7-0b8f6370ba30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1824c799a4041419b79251aabe01c2c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:41.898540 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a4f3a70f-7285-4b63-97f7-0b8f6370ba30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1824c799a4041419b79251aabe01c2c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:41.899457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a4f3a70f-7285-4b63-97f7-0b8f6370ba30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1824c799a4041419b79251aabe01c2c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:41.909199 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a4f3a70f-7285-4b63-97f7-0b8f6370ba30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1824c799a4041419b79251aabe01c2c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:41.930345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a4f3a70f-7285-4b63-97f7-0b8f6370ba30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1824c799a4041419b79251aabe01c2c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:41.937288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1073/2133] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:41 2022] DELETE /identity/v3/users/f1824c799a4041419b79251aabe01c2c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:42.208141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:42.209826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:42.227589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c2ccc-100e-4cd0-9e31-358328799bde admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:42.291305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04627589-4f62-42c7-9865-69cdea78cc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:42.291938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04627589-4f62-42c7-9865-69cdea78cc40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:42.292213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04627589-4f62-42c7-9865-69cdea78cc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:42.292487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04627589-4f62-42c7-9865-69cdea78cc40 admin admin] PATH_INFO: `/v3/projects/91ad9bd3e3cd404eaaa2c19f829eb27a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:42.293608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04627589-4f62-42c7-9865-69cdea78cc40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ad9bd3e3cd404eaaa2c19f829eb27a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:42.296053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04627589-4f62-42c7-9865-69cdea78cc40 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:42.342394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-04627589-4f62-42c7-9865-69cdea78cc40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ad9bd3e3cd404eaaa2c19f829eb27a', 'request_id': 'req-04627589-4f62-42c7-9865-69cdea78cc40'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:42.347445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:42.348760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-04627589-4f62-42c7-9865-69cdea78cc40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ad9bd3e3cd404eaaa2c19f829eb27a', 'request_id': 'req-04627589-4f62-42c7-9865-69cdea78cc40'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:42.357389 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-04627589-4f62-42c7-9865-69cdea78cc40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ad9bd3e3cd404eaaa2c19f829eb27a', 'request_id': 'req-04627589-4f62-42c7-9865-69cdea78cc40'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:42.373508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-04627589-4f62-42c7-9865-69cdea78cc40 admin admin] The token cache is being invalidate because project 91ad9bd3e3cd404eaaa2c19f829eb27a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:42.373808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-04627589-4f62-42c7-9865-69cdea78cc40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:42.374504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-04627589-4f62-42c7-9865-69cdea78cc40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:42.376035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1061/2134] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:42 2022] DELETE /identity/v3/projects/91ad9bd3e3cd404eaaa2c19f829eb27a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:42.380763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06356bd6-24cc-4f70-b237-8d4b9a693d6d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:42.381045 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06356bd6-24cc-4f70-b237-8d4b9a693d6d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:42.381318 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06356bd6-24cc-4f70-b237-8d4b9a693d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:42.401260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-06356bd6-24cc-4f70-b237-8d4b9a693d6d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:42.402503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-06356bd6-24cc-4f70-b237-8d4b9a693d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:42.464698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1074/2135] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:42.474780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5409e-8063-414b-93ee-d1927f03f595 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:42.475193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5409e-8063-414b-93ee-d1927f03f595 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:42.475540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5409e-8063-414b-93ee-d1927f03f595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:42.502271 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7fa5409e-8063-414b-93ee-d1927f03f595 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:42.503471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7fa5409e-8063-414b-93ee-d1927f03f595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:42.511703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1062/2136] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:42.516840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06356bd6-24cc-4f70-b237-8d4b9a693d6d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:42.527722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06356bd6-24cc-4f70-b237-8d4b9a693d6d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:42.532649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5cd63-fdbc-477d-b8c2-96b970390dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:42.533259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5cd63-fdbc-477d-b8c2-96b970390dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:42.533535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5cd63-fdbc-477d-b8c2-96b970390dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:42.533806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5cd63-fdbc-477d-b8c2-96b970390dc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:42.534555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5cd63-fdbc-477d-b8c2-96b970390dc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:25:42.534781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5cd63-fdbc-477d-b8c2-96b970390dc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:42.536379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5cd63-fdbc-477d-b8c2-96b970390dc6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:42.549520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1075/2137] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:25:42.553807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5409e-8063-414b-93ee-d1927f03f595 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:42.564970 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5409e-8063-414b-93ee-d1927f03f595 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:42.570041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38913e8-b9a1-46b8-aba1-257cc338a0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:42.570661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38913e8-b9a1-46b8-aba1-257cc338a0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:42.571303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38913e8-b9a1-46b8-aba1-257cc338a0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:42.571585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38913e8-b9a1-46b8-aba1-257cc338a0a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:42.572210 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38913e8-b9a1-46b8-aba1-257cc338a0a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:42.574101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38913e8-b9a1-46b8-aba1-257cc338a0a6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:42.603665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1063/2138] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:42 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:25:42.608567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5cd63-fdbc-477d-b8c2-96b970390dc6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:42.618561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5cd63-fdbc-477d-b8c2-96b970390dc6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:42.714960 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09cd52-44b4-434f-a6e8-c690e4eb42f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:42.716133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09cd52-44b4-434f-a6e8-c690e4eb42f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:42.716405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09cd52-44b4-434f-a6e8-c690e4eb42f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:42.716671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09cd52-44b4-434f-a6e8-c690e4eb42f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:42.717307 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09cd52-44b4-434f-a6e8-c690e4eb42f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:42.719337 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09cd52-44b4-434f-a6e8-c690e4eb42f4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:42.742881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1076/2139] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:42 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:42.747459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38913e8-b9a1-46b8-aba1-257cc338a0a6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:42.757980 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38913e8-b9a1-46b8-aba1-257cc338a0a6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:42.762859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7ae78-e9e1-4190-96fe-41a753b4616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:42.763449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7ae78-e9e1-4190-96fe-41a753b4616c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:42.763718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7ae78-e9e1-4190-96fe-41a753b4616c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:42.763979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7ae78-e9e1-4190-96fe-41a753b4616c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:42.764743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7ae78-e9e1-4190-96fe-41a753b4616c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:42.766368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7ae78-e9e1-4190-96fe-41a753b4616c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:42.780503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1064/2140] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:42.784736 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09cd52-44b4-434f-a6e8-c690e4eb42f4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:42.796153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09cd52-44b4-434f-a6e8-c690e4eb42f4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:42.803958 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f86a60-c842-40a1-9bb7-c5750b09a674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:42.804614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86a60-c842-40a1-9bb7-c5750b09a674 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:42.804897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86a60-c842-40a1-9bb7-c5750b09a674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:42.805184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f86a60-c842-40a1-9bb7-c5750b09a674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:42.805946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f86a60-c842-40a1-9bb7-c5750b09a674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:42.807589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f86a60-c842-40a1-9bb7-c5750b09a674 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:42.819348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1077/2141] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:42.823252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7ae78-e9e1-4190-96fe-41a753b4616c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:42.837585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7ae78-e9e1-4190-96fe-41a753b4616c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:42.845097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55211f9-6617-40fc-b781-294ef4dde310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:42.846058 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55211f9-6617-40fc-b781-294ef4dde310 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:42.846436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55211f9-6617-40fc-b781-294ef4dde310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:42.847304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55211f9-6617-40fc-b781-294ef4dde310 admin admin] PATH_INFO: `/v3/projects/a7e4f87baca042c79e511b8675e06f37/users/50f82d2f4e1f4c50bc9ba31ad913e30d/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:42.867107 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55211f9-6617-40fc-b781-294ef4dde310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7e4f87baca042c79e511b8675e06f37, user_id=50f82d2f4e1f4c50bc9ba31ad913e30d, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:42.869924 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55211f9-6617-40fc-b781-294ef4dde310 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:42.884643 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1065/2142] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:42 2022] PUT /identity/v3/projects/a7e4f87baca042c79e511b8675e06f37/users/50f82d2f4e1f4c50bc9ba31ad913e30d/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:25:42.896980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17db79-6917-4f96-8858-4d1637ceb0a3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:42.897173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17db79-6917-4f96-8858-4d1637ceb0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:42.898263 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17db79-6917-4f96-8858-4d1637ceb0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:42.913544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:42.939197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-4d17db79-6917-4f96-8858-4d1637ceb0a3 None None] MFA Rules not processed for user `50f82d2f4e1f4c50bc9ba31ad913e30d`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:42.940824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4d17db79-6917-4f96-8858-4d1637ceb0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:43.022483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1078/2143] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:43.035928 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55211f9-6617-40fc-b781-294ef4dde310 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:43.037632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e55211f9-6617-40fc-b781-294ef4dde310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:43.110255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55211f9-6617-40fc-b781-294ef4dde310 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:43.113945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e7a39-01dc-41d3-af61-8fc794a903dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:43.114573 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e7a39-01dc-41d3-af61-8fc794a903dd service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:43.114844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e7a39-01dc-41d3-af61-8fc794a903dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:43.115108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e7a39-01dc-41d3-af61-8fc794a903dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:43.124195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e7a39-01dc-41d3-af61-8fc794a903dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:43.125626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e7a39-01dc-41d3-af61-8fc794a903dd service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:43.134740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1066/2144] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:53.018973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=29 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:53.130703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=58,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:56.842130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17db79-6917-4f96-8858-4d1637ceb0a3 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:56.905879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17db79-6917-4f96-8858-4d1637ceb0a3 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:56.910674 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:56.911400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:56.911614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:56.911886 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] PATH_INFO: `/v3/users/50f82d2f4e1f4c50bc9ba31ad913e30d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:56.913021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f82d2f4e1f4c50bc9ba31ad913e30d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:56.914978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:56.979751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f82d2f4e1f4c50bc9ba31ad913e30d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:56.987423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f82d2f4e1f4c50bc9ba31ad913e30d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:56.988115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f82d2f4e1f4c50bc9ba31ad913e30d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:56.995471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f82d2f4e1f4c50bc9ba31ad913e30d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:57.007893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f82d2f4e1f4c50bc9ba31ad913e30d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:57.014319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1079/2145] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:25:56 2022] DELETE /identity/v3/users/50f82d2f4e1f4c50bc9ba31ad913e30d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:57.018760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b248d83-065a-4f05-ab47-67cdeaa6c6cb None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:57.019219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b248d83-065a-4f05-ab47-67cdeaa6c6cb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:57.019310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b248d83-065a-4f05-ab47-67cdeaa6c6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:57.038757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7b248d83-065a-4f05-ab47-67cdeaa6c6cb None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:57.039955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7b248d83-065a-4f05-ab47-67cdeaa6c6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:57.101347 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1067/2146] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:57.111545 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:57.113000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:57.170261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3167ae69-e450-4390-b570-999bf20cc057 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:57.174415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d73a8f-10dd-4af7-bf9d-4dddf69c25b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:57.175020 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d73a8f-10dd-4af7-bf9d-4dddf69c25b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:57.175297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d73a8f-10dd-4af7-bf9d-4dddf69c25b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:57.175565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d73a8f-10dd-4af7-bf9d-4dddf69c25b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:57.185517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d73a8f-10dd-4af7-bf9d-4dddf69c25b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:57.187086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d73a8f-10dd-4af7-bf9d-4dddf69c25b9 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:57.197267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1080/2147] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:25:57.508727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b248d83-065a-4f05-ab47-67cdeaa6c6cb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:57.510365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7b248d83-065a-4f05-ab47-67cdeaa6c6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:57.535906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b248d83-065a-4f05-ab47-67cdeaa6c6cb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:57.541780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:57.542792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:57.543363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:57.544011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85 admin admin] PATH_INFO: `/v3/projects/a7e4f87baca042c79e511b8675e06f37` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:57.545286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7e4f87baca042c79e511b8675e06f37)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:57.547561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:57.623785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7e4f87baca042c79e511b8675e06f37', 'request_id': 'req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:57.630238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7e4f87baca042c79e511b8675e06f37', 'request_id': 'req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:57.637369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7e4f87baca042c79e511b8675e06f37', 'request_id': 'req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:57.654296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85 admin admin] The token cache is being invalidate because project a7e4f87baca042c79e511b8675e06f37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:25:57.654451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:57.655081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-af7aa53f-81af-41a7-a1aa-f9e97aa49a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:25:57.656824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1068/2148] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:25:57 2022] DELETE /identity/v3/projects/a7e4f87baca042c79e511b8675e06f37 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:57.661856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353cff7-937a-4472-bd28-a13e43ef0ff9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:57.662369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353cff7-937a-4472-bd28-a13e43ef0ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:57.663516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353cff7-937a-4472-bd28-a13e43ef0ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:57.691618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3353cff7-937a-4472-bd28-a13e43ef0ff9 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:57.693380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3353cff7-937a-4472-bd28-a13e43ef0ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:57.718838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:57.811992 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc04c4d-3b6a-4bb9-9099-1741f42ed888 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:57.811992 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc04c4d-3b6a-4bb9-9099-1741f42ed888 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:57.811992 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc04c4d-3b6a-4bb9-9099-1741f42ed888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:57.819954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1081/2149] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:57.870614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fdc04c4d-3b6a-4bb9-9099-1741f42ed888 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:57.872332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fdc04c4d-3b6a-4bb9-9099-1741f42ed888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:57.881170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1069/2150] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:57.885588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353cff7-937a-4472-bd28-a13e43ef0ff9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:57.915774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353cff7-937a-4472-bd28-a13e43ef0ff9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:57.930637 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046ae5b-0629-445e-827e-4cd1b7bda061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:57.931266 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046ae5b-0629-445e-827e-4cd1b7bda061 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:57.931541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046ae5b-0629-445e-827e-4cd1b7bda061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:57.933105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046ae5b-0629-445e-827e-4cd1b7bda061 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:57.934320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046ae5b-0629-445e-827e-4cd1b7bda061 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:25:57.934648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046ae5b-0629-445e-827e-4cd1b7bda061 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:57.936803 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046ae5b-0629-445e-827e-4cd1b7bda061 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:57.967091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1082/2151] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:25:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:25:57.967752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc04c4d-3b6a-4bb9-9099-1741f42ed888 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:57.984601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc04c4d-3b6a-4bb9-9099-1741f42ed888 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:57.999049 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce52bdd-fdff-42f7-b27e-5c7224dca6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:57.999670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce52bdd-fdff-42f7-b27e-5c7224dca6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:57.999944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce52bdd-fdff-42f7-b27e-5c7224dca6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:58.000213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce52bdd-fdff-42f7-b27e-5c7224dca6d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:58.000845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce52bdd-fdff-42f7-b27e-5c7224dca6d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:58.002802 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce52bdd-fdff-42f7-b27e-5c7224dca6d9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:58.057301 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1070/2152] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:57 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:58.058712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046ae5b-0629-445e-827e-4cd1b7bda061 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:58.070681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046ae5b-0629-445e-827e-4cd1b7bda061 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:58.139268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd23688-86bd-46ba-895a-fb0ef14323da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:58.140445 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd23688-86bd-46ba-895a-fb0ef14323da admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:58.140830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd23688-86bd-46ba-895a-fb0ef14323da admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:58.141353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd23688-86bd-46ba-895a-fb0ef14323da admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:58.142762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd23688-86bd-46ba-895a-fb0ef14323da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:58.145340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd23688-86bd-46ba-895a-fb0ef14323da admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:58.176394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1083/2153] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:58 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:25:58.180907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce52bdd-fdff-42f7-b27e-5c7224dca6d9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:58.194738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce52bdd-fdff-42f7-b27e-5c7224dca6d9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:58.200758 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79731454-de65-4493-9e64-57c19e099943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:58.201479 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79731454-de65-4493-9e64-57c19e099943 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:58.201832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79731454-de65-4493-9e64-57c19e099943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:58.202236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79731454-de65-4493-9e64-57c19e099943 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:58.202950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79731454-de65-4493-9e64-57c19e099943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:58.204604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79731454-de65-4493-9e64-57c19e099943 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:58.220141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1071/2154] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:58.221367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd23688-86bd-46ba-895a-fb0ef14323da admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:58.232050 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd23688-86bd-46ba-895a-fb0ef14323da admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:58.237479 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f345376-0cd2-47be-bd40-c0930d6172ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:58.238104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f345376-0cd2-47be-bd40-c0930d6172ce admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:58.238704 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f345376-0cd2-47be-bd40-c0930d6172ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:58.239107 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f345376-0cd2-47be-bd40-c0930d6172ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:58.239768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f345376-0cd2-47be-bd40-c0930d6172ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:58.241897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f345376-0cd2-47be-bd40-c0930d6172ce admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:58.254622 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1084/2155] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:58.259632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79731454-de65-4493-9e64-57c19e099943 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:58.272026 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79731454-de65-4493-9e64-57c19e099943 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:58.279171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141ea0a-2009-45a8-a228-e67cbf6738c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:58.281356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141ea0a-2009-45a8-a228-e67cbf6738c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:58.281846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141ea0a-2009-45a8-a228-e67cbf6738c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:58.282211 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141ea0a-2009-45a8-a228-e67cbf6738c4 admin admin] PATH_INFO: `/v3/projects/b819bacc1949451b8bc3e9488cf5a354/users/96ae83f39e934ed19467000a8328346a/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:58.307041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141ea0a-2009-45a8-a228-e67cbf6738c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b819bacc1949451b8bc3e9488cf5a354, user_id=96ae83f39e934ed19467000a8328346a, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:58.307041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141ea0a-2009-45a8-a228-e67cbf6738c4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:58.326162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1072/2156] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:58 2022] PUT /identity/v3/projects/b819bacc1949451b8bc3e9488cf5a354/users/96ae83f39e934ed19467000a8328346a/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:25:58.334735 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556bbc3-8301-4a0e-8589-e17a8cc1cc20 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:58.335024 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556bbc3-8301-4a0e-8589-e17a8cc1cc20 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:58.335301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556bbc3-8301-4a0e-8589-e17a8cc1cc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:58.367898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e556bbc3-8301-4a0e-8589-e17a8cc1cc20 None None] MFA Rules not processed for user `96ae83f39e934ed19467000a8328346a`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:58.369100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e556bbc3-8301-4a0e-8589-e17a8cc1cc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:58.457640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1085/2157] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:25:58.462113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141ea0a-2009-45a8-a228-e67cbf6738c4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:58.494806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:58.546812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141ea0a-2009-45a8-a228-e67cbf6738c4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:58.551412 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10843f9-6e45-4863-9542-9094e8689d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:58.552003 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10843f9-6e45-4863-9542-9094e8689d7e admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:58.552282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10843f9-6e45-4863-9542-9094e8689d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:58.552550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10843f9-6e45-4863-9542-9094e8689d7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:58.553167 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10843f9-6e45-4863-9542-9094e8689d7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:58.555080 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10843f9-6e45-4863-9542-9094e8689d7e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:58.596697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1073/2158] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:58 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:25:58.597879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556bbc3-8301-4a0e-8589-e17a8cc1cc20 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:58.610319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556bbc3-8301-4a0e-8589-e17a8cc1cc20 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:58.630749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:58.681086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fd45e-fea7-49be-9ae8-9781d8e749d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:58.681686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fd45e-fea7-49be-9ae8-9781d8e749d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:58.681963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fd45e-fea7-49be-9ae8-9781d8e749d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:58.682265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fd45e-fea7-49be-9ae8-9781d8e749d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:58.682898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028fd45e-fea7-49be-9ae8-9781d8e749d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:58.684995 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028fd45e-fea7-49be-9ae8-9781d8e749d9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:58.725368 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1086/2159] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:58 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:25:58.729911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10843f9-6e45-4863-9542-9094e8689d7e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:58.742767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10843f9-6e45-4863-9542-9094e8689d7e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:58.747954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccabad6-8281-4859-8f44-e937d6376585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:58.748741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccabad6-8281-4859-8f44-e937d6376585 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:58.749054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccabad6-8281-4859-8f44-e937d6376585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:58.749302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccabad6-8281-4859-8f44-e937d6376585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:58.749988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccabad6-8281-4859-8f44-e937d6376585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:58.751617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccabad6-8281-4859-8f44-e937d6376585 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:58.766119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1074/2160] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:58.768647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fd45e-fea7-49be-9ae8-9781d8e749d9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:58.779543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fd45e-fea7-49be-9ae8-9781d8e749d9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:58.784527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c30c9-4752-44e3-b0e6-c363c34b0950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:58.785123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c30c9-4752-44e3-b0e6-c363c34b0950 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:58.785394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c30c9-4752-44e3-b0e6-c363c34b0950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:58.785661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c30c9-4752-44e3-b0e6-c363c34b0950 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:58.787837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c30c9-4752-44e3-b0e6-c363c34b0950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:58.788158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c30c9-4752-44e3-b0e6-c363c34b0950 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:58.803613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1087/2161] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:25:58.805172 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccabad6-8281-4859-8f44-e937d6376585 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:58.815612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccabad6-8281-4859-8f44-e937d6376585 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:58.821720 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667565f0-b0c5-445c-905f-dc998f20e4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:58.822409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667565f0-b0c5-445c-905f-dc998f20e4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:58.823624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667565f0-b0c5-445c-905f-dc998f20e4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:58.823624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667565f0-b0c5-445c-905f-dc998f20e4e5 admin admin] PATH_INFO: `/v3/projects/f1886c4475274b5481800682ea4cd98f/users/0481760bc7674b24a6244de0391f33b3/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:58.842641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667565f0-b0c5-445c-905f-dc998f20e4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1886c4475274b5481800682ea4cd98f, user_id=0481760bc7674b24a6244de0391f33b3, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:58.846612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667565f0-b0c5-445c-905f-dc998f20e4e5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:58.862229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1075/2162] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:58 2022] PUT /identity/v3/projects/f1886c4475274b5481800682ea4cd98f/users/0481760bc7674b24a6244de0391f33b3/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:58.872569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af0602-efb0-40b3-b591-c1b71822f39f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:58.873143 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af0602-efb0-40b3-b591-c1b71822f39f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:58.873589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af0602-efb0-40b3-b591-c1b71822f39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:58.908604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-47af0602-efb0-40b3-b591-c1b71822f39f None None] MFA Rules not processed for user `0481760bc7674b24a6244de0391f33b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:58.910036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-47af0602-efb0-40b3-b591-c1b71822f39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:58.979616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1088/2163] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:58.985299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667565f0-b0c5-445c-905f-dc998f20e4e5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:59.062613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667565f0-b0c5-445c-905f-dc998f20e4e5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:59.062613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd10ae-6a1a-4304-99fe-ebf3a6606045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:59.062613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd10ae-6a1a-4304-99fe-ebf3a6606045 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:59.062613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd10ae-6a1a-4304-99fe-ebf3a6606045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:59.063635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd10ae-6a1a-4304-99fe-ebf3a6606045 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:59.063635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd10ae-6a1a-4304-99fe-ebf3a6606045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:59.063635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd10ae-6a1a-4304-99fe-ebf3a6606045 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:59.100579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1076/2164] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:25:58 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:25:59.102138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af0602-efb0-40b3-b591-c1b71822f39f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:59.114308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af0602-efb0-40b3-b591-c1b71822f39f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:59.217492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fedcb3-44bc-494f-b819-03d14b3fe331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:59.218227 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fedcb3-44bc-494f-b819-03d14b3fe331 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:59.218565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fedcb3-44bc-494f-b819-03d14b3fe331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:59.219021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fedcb3-44bc-494f-b819-03d14b3fe331 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:59.219649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fedcb3-44bc-494f-b819-03d14b3fe331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:59.221755 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fedcb3-44bc-494f-b819-03d14b3fe331 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:59.258326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1089/2165] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:25:59 2022] POST /identity/v3/users => generated 298 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:25:59.259577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd10ae-6a1a-4304-99fe-ebf3a6606045 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:59.272620 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd10ae-6a1a-4304-99fe-ebf3a6606045 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:59.278754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1df2e-5c17-4735-a35c-839487c1df54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:59.279702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1df2e-5c17-4735-a35c-839487c1df54 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:59.279996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1df2e-5c17-4735-a35c-839487c1df54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:59.280267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1df2e-5c17-4735-a35c-839487c1df54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:59.280942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1df2e-5c17-4735-a35c-839487c1df54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:59.282736 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1df2e-5c17-4735-a35c-839487c1df54 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:59.307769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1077/2166] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:25:59.309215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fedcb3-44bc-494f-b819-03d14b3fe331 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:59.327443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fedcb3-44bc-494f-b819-03d14b3fe331 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:59.332874 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40079c00-c268-45f1-9b47-2c568befe06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:59.333683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40079c00-c268-45f1-9b47-2c568befe06a admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:59.334060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40079c00-c268-45f1-9b47-2c568befe06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:59.334384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40079c00-c268-45f1-9b47-2c568befe06a admin admin] PATH_INFO: `/v3/projects/1307defc25444525ba4f7d3ddd7cc4fa/users/09e58d04403a4a22836944f3d892a3d3/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:59.349792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:59.355043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40079c00-c268-45f1-9b47-2c568befe06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1307defc25444525ba4f7d3ddd7cc4fa, user_id=09e58d04403a4a22836944f3d892a3d3, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:59.359357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40079c00-c268-45f1-9b47-2c568befe06a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:59.373104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1090/2167] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:59 2022] PUT /identity/v3/projects/1307defc25444525ba4f7d3ddd7cc4fa/users/09e58d04403a4a22836944f3d892a3d3/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:59.377065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1df2e-5c17-4735-a35c-839487c1df54 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:59.387096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1df2e-5c17-4735-a35c-839487c1df54 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:59.428678 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:25:59.445239 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ae119-1780-4a68-9ad1-d0dbf684e518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:59.445840 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ae119-1780-4a68-9ad1-d0dbf684e518 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:59.446141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ae119-1780-4a68-9ad1-d0dbf684e518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:59.446414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ae119-1780-4a68-9ad1-d0dbf684e518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:59.447092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ae119-1780-4a68-9ad1-d0dbf684e518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:59.448673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ae119-1780-4a68-9ad1-d0dbf684e518 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:59.459239 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1078/2168] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:25:59.463370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40079c00-c268-45f1-9b47-2c568befe06a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:59.474518 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40079c00-c268-45f1-9b47-2c568befe06a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:59.479474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef467285-5878-4192-9ae7-e90c4e4e629a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:59.480121 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef467285-5878-4192-9ae7-e90c4e4e629a admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:59.480655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef467285-5878-4192-9ae7-e90c4e4e629a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:59.480925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef467285-5878-4192-9ae7-e90c4e4e629a admin admin] PATH_INFO: `/v3/projects/1307defc25444525ba4f7d3ddd7cc4fa/users/09e58d04403a4a22836944f3d892a3d3/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:59.488998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef467285-5878-4192-9ae7-e90c4e4e629a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1307defc25444525ba4f7d3ddd7cc4fa, user_id=09e58d04403a4a22836944f3d892a3d3, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:59.488998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef467285-5878-4192-9ae7-e90c4e4e629a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:59.498828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1091/2169] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:25:59 2022] PUT /identity/v3/projects/1307defc25444525ba4f7d3ddd7cc4fa/users/09e58d04403a4a22836944f3d892a3d3/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:25:59.510399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bd2f3-b367-4090-bd07-a50d51ed917f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:59.510683 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bd2f3-b367-4090-bd07-a50d51ed917f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:59.510959 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48bd2f3-b367-4090-bd07-a50d51ed917f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:59.539225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d48bd2f3-b367-4090-bd07-a50d51ed917f None None] MFA Rules not processed for user `09e58d04403a4a22836944f3d892a3d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:25:59.540456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d48bd2f3-b367-4090-bd07-a50d51ed917f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:59.611353 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1079/2170] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:25:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:25:59.623224 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef467285-5878-4192-9ae7-e90c4e4e629a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:25:59.625026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ef467285-5878-4192-9ae7-e90c4e4e629a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:25:59.685581 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef467285-5878-4192-9ae7-e90c4e4e629a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:25:59.689437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9cde1-7cec-4d53-b457-6964e1adb932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:25:59.690059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9cde1-7cec-4d53-b457-6964e1adb932 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:25:59.690370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9cde1-7cec-4d53-b457-6964e1adb932 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:25:59.690678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c9cde1-7cec-4d53-b457-6964e1adb932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:25:59.755006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9cde1-7cec-4d53-b457-6964e1adb932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:25:59.756437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c9cde1-7cec-4d53-b457-6964e1adb932 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:25:59.765714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1092/2171] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:25:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:26:09.608462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=43 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:09.761164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:11.827511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48bd2f3-b367-4090-bd07-a50d51ed917f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:11.836855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48bd2f3-b367-4090-bd07-a50d51ed917f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:11.841007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69491565-a592-4ef2-8942-881cfa4b63dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:11.841590 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69491565-a592-4ef2-8942-881cfa4b63dc service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:11.841860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69491565-a592-4ef2-8942-881cfa4b63dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:11.842157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69491565-a592-4ef2-8942-881cfa4b63dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:11.903393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69491565-a592-4ef2-8942-881cfa4b63dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:11.904842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69491565-a592-4ef2-8942-881cfa4b63dc service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:11.914153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1080/2172] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:26:14.878513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9cde1-7cec-4d53-b457-6964e1adb932 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:14.887026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c9cde1-7cec-4d53-b457-6964e1adb932 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:14.890636 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06df6a4-ae62-43fe-91e5-5b0985b1e5ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:14.891229 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06df6a4-ae62-43fe-91e5-5b0985b1e5ce service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:14.891508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06df6a4-ae62-43fe-91e5-5b0985b1e5ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:14.891779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06df6a4-ae62-43fe-91e5-5b0985b1e5ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:14.900514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06df6a4-ae62-43fe-91e5-5b0985b1e5ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:14.901923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06df6a4-ae62-43fe-91e5-5b0985b1e5ce service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:14.911722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1093/2173] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:26:21.910118 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=28 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:22.409083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69491565-a592-4ef2-8942-881cfa4b63dc service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:22.410821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-69491565-a592-4ef2-8942-881cfa4b63dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:22.513665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69491565-a592-4ef2-8942-881cfa4b63dc service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:22.519222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:22.519978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:22.520248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:22.520670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] PATH_INFO: `/v3/users/8209748fe1f344caa843b316ad2db2d7` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:22.522848 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8209748fe1f344caa843b316ad2db2d7)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:22.525506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:22.587287 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8209748fe1f344caa843b316ad2db2d7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:22.587908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8209748fe1f344caa843b316ad2db2d7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:22.596694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8209748fe1f344caa843b316ad2db2d7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:22.608999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8209748fe1f344caa843b316ad2db2d7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:22.622915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8209748fe1f344caa843b316ad2db2d7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:22.631328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1081/2174] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:26:22 2022] DELETE /identity/v3/users/8209748fe1f344caa843b316ad2db2d7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:26:22.638521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c240ffc-b39d-4272-9f41-63e09a5872e3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:22.641171 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c240ffc-b39d-4272-9f41-63e09a5872e3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:22.641461 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c240ffc-b39d-4272-9f41-63e09a5872e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:22.669624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8c240ffc-b39d-4272-9f41-63e09a5872e3 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:22.671476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8c240ffc-b39d-4272-9f41-63e09a5872e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:22.744484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1094/2175] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:22.755381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:22.756928 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:22.816405 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def80b3-c3c2-4069-8f85-23a96b8e5544 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:22.820165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bacbc-a9fd-47aa-93e5-9ab1ae216597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:22.820771 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bacbc-a9fd-47aa-93e5-9ab1ae216597 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:22.821047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bacbc-a9fd-47aa-93e5-9ab1ae216597 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:22.821319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bacbc-a9fd-47aa-93e5-9ab1ae216597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:22.831083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bacbc-a9fd-47aa-93e5-9ab1ae216597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:22.832558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bacbc-a9fd-47aa-93e5-9ab1ae216597 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:22.842852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1082/2176] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:26:23.173443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c240ffc-b39d-4272-9f41-63e09a5872e3 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:23.174958 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8c240ffc-b39d-4272-9f41-63e09a5872e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:23.197353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c240ffc-b39d-4272-9f41-63e09a5872e3 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:23.204423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1795fd29-367d-47b9-a8d4-32d9fa9190d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:23.205734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795fd29-367d-47b9-a8d4-32d9fa9190d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:23.206291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795fd29-367d-47b9-a8d4-32d9fa9190d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:23.206813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795fd29-367d-47b9-a8d4-32d9fa9190d6 admin admin] PATH_INFO: `/v3/users/96ae83f39e934ed19467000a8328346a` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:23.208325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795fd29-367d-47b9-a8d4-32d9fa9190d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96ae83f39e934ed19467000a8328346a)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:23.210657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1795fd29-367d-47b9-a8d4-32d9fa9190d6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:23.221984 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bacbc-a9fd-47aa-93e5-9ab1ae216597 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:23.223469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-859bacbc-a9fd-47aa-93e5-9ab1ae216597 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:23.279630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bacbc-a9fd-47aa-93e5-9ab1ae216597 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:23.284696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1795fd29-367d-47b9-a8d4-32d9fa9190d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96ae83f39e934ed19467000a8328346a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:23.286230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fe004-92d9-4595-9203-8bcedcd305f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:23.287281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fe004-92d9-4595-9203-8bcedcd305f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:23.287572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fe004-92d9-4595-9203-8bcedcd305f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:23.287835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fe004-92d9-4595-9203-8bcedcd305f3 admin admin] PATH_INFO: `/v3/projects/6675c176d490466bac2697801fcfe5e8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:23.289044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fe004-92d9-4595-9203-8bcedcd305f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6675c176d490466bac2697801fcfe5e8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:23.291179 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fe004-92d9-4595-9203-8bcedcd305f3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:23.295620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1795fd29-367d-47b9-a8d4-32d9fa9190d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96ae83f39e934ed19467000a8328346a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:23.298492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1795fd29-367d-47b9-a8d4-32d9fa9190d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96ae83f39e934ed19467000a8328346a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:23.321277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1795fd29-367d-47b9-a8d4-32d9fa9190d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96ae83f39e934ed19467000a8328346a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:23.326667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:23.343657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1795fd29-367d-47b9-a8d4-32d9fa9190d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96ae83f39e934ed19467000a8328346a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:23.352043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1095/2177] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:26:23 2022] DELETE /identity/v3/users/96ae83f39e934ed19467000a8328346a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:23.358353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8a939-2b8c-4d14-97db-da48c76522ba None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:23.358856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8a939-2b8c-4d14-97db-da48c76522ba None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:23.359374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8a939-2b8c-4d14-97db-da48c76522ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:23.371082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd8fe004-92d9-4595-9203-8bcedcd305f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6675c176d490466bac2697801fcfe5e8', 'request_id': 'req-fd8fe004-92d9-4595-9203-8bcedcd305f3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:23.378312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd8fe004-92d9-4595-9203-8bcedcd305f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6675c176d490466bac2697801fcfe5e8', 'request_id': 'req-fd8fe004-92d9-4595-9203-8bcedcd305f3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:23.384636 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6ff8a939-2b8c-4d14-97db-da48c76522ba None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:23.385842 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6ff8a939-2b8c-4d14-97db-da48c76522ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:23.387277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd8fe004-92d9-4595-9203-8bcedcd305f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6675c176d490466bac2697801fcfe5e8', 'request_id': 'req-fd8fe004-92d9-4595-9203-8bcedcd305f3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:23.411871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd8fe004-92d9-4595-9203-8bcedcd305f3 admin admin] The token cache is being invalidate because project 6675c176d490466bac2697801fcfe5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:26:23.412160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd8fe004-92d9-4595-9203-8bcedcd305f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:23.413476 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd8fe004-92d9-4595-9203-8bcedcd305f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:23.415669 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1083/2178] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:26:23 2022] DELETE /identity/v3/projects/6675c176d490466bac2697801fcfe5e8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:23.420459 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a33e1-e9ff-443e-9232-bbfdd74efa8c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:23.420750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a33e1-e9ff-443e-9232-bbfdd74efa8c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:23.421028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480a33e1-e9ff-443e-9232-bbfdd74efa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:23.454552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-480a33e1-e9ff-443e-9232-bbfdd74efa8c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:23.457217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-480a33e1-e9ff-443e-9232-bbfdd74efa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:23.463144 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:23.482391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1096/2179] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:23.499780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8a939-2b8c-4d14-97db-da48c76522ba None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:23.506499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6ff8a939-2b8c-4d14-97db-da48c76522ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:23.611094 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1084/2180] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:23.615371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8a939-2b8c-4d14-97db-da48c76522ba None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:23.622765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff520dac-543a-4311-b65f-477dbbacea7e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:23.623044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff520dac-543a-4311-b65f-477dbbacea7e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:23.623317 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff520dac-543a-4311-b65f-477dbbacea7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:23.627617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad79a6-8df0-4649-b05e-0d87117453f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:23.628794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad79a6-8df0-4649-b05e-0d87117453f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:23.629068 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad79a6-8df0-4649-b05e-0d87117453f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:23.629338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad79a6-8df0-4649-b05e-0d87117453f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:23.644622 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ff520dac-543a-4311-b65f-477dbbacea7e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:23.646279 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ff520dac-543a-4311-b65f-477dbbacea7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:23.654592 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad79a6-8df0-4649-b05e-0d87117453f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:23.656438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad79a6-8df0-4649-b05e-0d87117453f9 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:23.658917 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1085/2181] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:23.663996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff520dac-543a-4311-b65f-477dbbacea7e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:23.678345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1097/2182] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:26:23.696076 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff520dac-543a-4311-b65f-477dbbacea7e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:23.708897 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19274e-cde4-441b-bd43-58c749d5255e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:23.710139 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19274e-cde4-441b-bd43-58c749d5255e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:23.710967 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19274e-cde4-441b-bd43-58c749d5255e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:23.711431 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19274e-cde4-441b-bd43-58c749d5255e admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:23.714010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19274e-cde4-441b-bd43-58c749d5255e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:26:23.715216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19274e-cde4-441b-bd43-58c749d5255e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:23.719697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19274e-cde4-441b-bd43-58c749d5255e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:23.765732 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad79a6-8df0-4649-b05e-0d87117453f9 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:23.774424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1086/2183] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:26:23.785560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad79a6-8df0-4649-b05e-0d87117453f9 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:23.792821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84b1b5-4408-49ae-b683-d0a328e7bfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:23.794405 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84b1b5-4408-49ae-b683-d0a328e7bfcf admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:23.795416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84b1b5-4408-49ae-b683-d0a328e7bfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:23.796041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84b1b5-4408-49ae-b683-d0a328e7bfcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:23.797045 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84b1b5-4408-49ae-b683-d0a328e7bfcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:23.799738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84b1b5-4408-49ae-b683-d0a328e7bfcf admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:23.847600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1098/2184] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:26:23 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:23.851907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19274e-cde4-441b-bd43-58c749d5255e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:23.875290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19274e-cde4-441b-bd43-58c749d5255e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:23.998309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6caebe-099e-450a-81b4-fd20d4e4a967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:23.998922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6caebe-099e-450a-81b4-fd20d4e4a967 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:23.999329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6caebe-099e-450a-81b4-fd20d4e4a967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:23.999550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6caebe-099e-450a-81b4-fd20d4e4a967 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:24.000267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6caebe-099e-450a-81b4-fd20d4e4a967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:24.003480 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6caebe-099e-450a-81b4-fd20d4e4a967 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:24.056767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84b1b5-4408-49ae-b683-d0a328e7bfcf admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:24.057038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1087/2185] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:26:23 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:24.076988 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84b1b5-4408-49ae-b683-d0a328e7bfcf admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:24.091240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24483cd2-40ec-4f75-a258-8188dad1eb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:24.092082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24483cd2-40ec-4f75-a258-8188dad1eb7e admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:24.092530 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24483cd2-40ec-4f75-a258-8188dad1eb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:24.092952 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24483cd2-40ec-4f75-a258-8188dad1eb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:24.093756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24483cd2-40ec-4f75-a258-8188dad1eb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:24.096147 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24483cd2-40ec-4f75-a258-8188dad1eb7e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:24.121383 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1099/2186] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:26:24.132489 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6caebe-099e-450a-81b4-fd20d4e4a967 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:24.144701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6caebe-099e-450a-81b4-fd20d4e4a967 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:24.150988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd7f55-a3d6-47ff-b888-d68c46237c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:24.151829 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd7f55-a3d6-47ff-b888-d68c46237c19 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:24.152226 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd7f55-a3d6-47ff-b888-d68c46237c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:24.152498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd7f55-a3d6-47ff-b888-d68c46237c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:24.153830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd7f55-a3d6-47ff-b888-d68c46237c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:24.155291 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd7f55-a3d6-47ff-b888-d68c46237c19 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:24.163570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24483cd2-40ec-4f75-a258-8188dad1eb7e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:24.165453 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-24483cd2-40ec-4f75-a258-8188dad1eb7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:24.173323 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1088/2187] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:26:24.177644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd7f55-a3d6-47ff-b888-d68c46237c19 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:24.180159 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24483cd2-40ec-4f75-a258-8188dad1eb7e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:24.188279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:24.188967 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:24.189239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:24.189507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] PATH_INFO: `/v3/projects/b819bacc1949451b8bc3e9488cf5a354` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:24.191649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b819bacc1949451b8bc3e9488cf5a354)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:24.193335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:24.194387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:24.200397 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd7f55-a3d6-47ff-b888-d68c46237c19 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:24.209810 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00372eac-2cf4-41fb-9bbf-bb001818eb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:24.210744 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00372eac-2cf4-41fb-9bbf-bb001818eb9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:24.211217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00372eac-2cf4-41fb-9bbf-bb001818eb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:24.211674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00372eac-2cf4-41fb-9bbf-bb001818eb9b admin admin] PATH_INFO: `/v3/projects/d2fe812c4a7745d181fbb91a4bfafc36/users/2106efd899d64217ace2e0d71ceea1be/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:24.232167 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00372eac-2cf4-41fb-9bbf-bb001818eb9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2fe812c4a7745d181fbb91a4bfafc36, user_id=2106efd899d64217ace2e0d71ceea1be, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:24.235418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00372eac-2cf4-41fb-9bbf-bb001818eb9b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:24.251703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1089/2188] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:26:24 2022] PUT /identity/v3/projects/d2fe812c4a7745d181fbb91a4bfafc36/users/2106efd899d64217ace2e0d71ceea1be/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:26:24.260406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b819bacc1949451b8bc3e9488cf5a354', 'request_id': 'req-6b624a70-0203-4690-84d6-77360e437d80'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.263054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392b5d0-c68f-4d57-bdb3-2a2155f811b6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:24.263495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392b5d0-c68f-4d57-bdb3-2a2155f811b6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:24.264274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392b5d0-c68f-4d57-bdb3-2a2155f811b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:24.274385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b819bacc1949451b8bc3e9488cf5a354', 'request_id': 'req-6b624a70-0203-4690-84d6-77360e437d80'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.283560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b819bacc1949451b8bc3e9488cf5a354', 'request_id': 'req-6b624a70-0203-4690-84d6-77360e437d80'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.298925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4392b5d0-c68f-4d57-bdb3-2a2155f811b6 None None] MFA Rules not processed for user `2106efd899d64217ace2e0d71ceea1be`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:24.300141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4392b5d0-c68f-4d57-bdb3-2a2155f811b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:24.301592 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] The token cache is being invalidate because project b819bacc1949451b8bc3e9488cf5a354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:26:24.302032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.302882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.305036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1100/2189] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:26:24 2022] DELETE /identity/v3/projects/b819bacc1949451b8bc3e9488cf5a354 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:24.309271 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:24.311038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:24.384130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:24.404604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b624a70-0203-4690-84d6-77360e437d80 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:24.408869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1090/2190] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:26:24.412661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:24.413588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:24.413867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:24.414175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] PATH_INFO: `/v3/users/0481760bc7674b24a6244de0391f33b3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:24.415377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0481760bc7674b24a6244de0391f33b3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:24.417529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:24.424607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392b5d0-c68f-4d57-bdb3-2a2155f811b6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:24.426451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4392b5d0-c68f-4d57-bdb3-2a2155f811b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:24.483791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0481760bc7674b24a6244de0391f33b3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.492372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0481760bc7674b24a6244de0391f33b3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.493458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0481760bc7674b24a6244de0391f33b3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.507516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0481760bc7674b24a6244de0391f33b3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.529849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392b5d0-c68f-4d57-bdb3-2a2155f811b6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:24.535771 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ccc4a-c2a9-4161-9224-8b7a62d22edf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:24.536660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ccc4a-c2a9-4161-9224-8b7a62d22edf service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:24.537033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ccc4a-c2a9-4161-9224-8b7a62d22edf service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:24.537314 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ccc4a-c2a9-4161-9224-8b7a62d22edf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:24.539355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dd2ccc4a-c2a9-4161-9224-8b7a62d22edf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:24.539471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0481760bc7674b24a6244de0391f33b3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.549425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1101/2191] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:26:24 2022] DELETE /identity/v3/users/0481760bc7674b24a6244de0391f33b3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:24.554805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ccc4a-c2a9-4161-9224-8b7a62d22edf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:24.557066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ccc4a-c2a9-4161-9224-8b7a62d22edf service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:24.578099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1091/2192] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:26:24.597191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:24.598826 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:24.686841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15daa2e4-6c70-4c75-81e6-0fddd8f896cc admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:24.692787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:24.694109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:24.694936 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:24.695279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] PATH_INFO: `/v3/users/2106efd899d64217ace2e0d71ceea1be` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:24.697852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2106efd899d64217ace2e0d71ceea1be)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:24.701853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:24.811768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2106efd899d64217ace2e0d71ceea1be'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.828438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2106efd899d64217ace2e0d71ceea1be'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.829868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2106efd899d64217ace2e0d71ceea1be'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.842440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2106efd899d64217ace2e0d71ceea1be'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.869756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:24.880696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2106efd899d64217ace2e0d71ceea1be'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:24.889098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1102/2193] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:26:24 2022] DELETE /identity/v3/users/2106efd899d64217ace2e0d71ceea1be => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:24.893842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da3b0f-7007-4ceb-875c-f88e5e66ae3b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:24.894185 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da3b0f-7007-4ceb-875c-f88e5e66ae3b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:24.894935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da3b0f-7007-4ceb-875c-f88e5e66ae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:24.926539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-81da3b0f-7007-4ceb-875c-f88e5e66ae3b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:24.931543 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-81da3b0f-7007-4ceb-875c-f88e5e66ae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:25.030415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1092/2194] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:25.041369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:25.043028 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:25.065514 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da3b0f-7007-4ceb-875c-f88e5e66ae3b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:25.066990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-81da3b0f-7007-4ceb-875c-f88e5e66ae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:25.092376 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da3b0f-7007-4ceb-875c-f88e5e66ae3b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:25.118628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086299d-95d8-4925-9150-62cef4cadbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:25.120508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086299d-95d8-4925-9150-62cef4cadbbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:25.123017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086299d-95d8-4925-9150-62cef4cadbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:25.125482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086299d-95d8-4925-9150-62cef4cadbbc admin admin] PATH_INFO: `/v3/projects/f1886c4475274b5481800682ea4cd98f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:25.128414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086299d-95d8-4925-9150-62cef4cadbbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1886c4475274b5481800682ea4cd98f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:25.131311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086299d-95d8-4925-9150-62cef4cadbbc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:25.153365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:25.180775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede9ec5-47cd-460f-8a61-f83f8a57c95b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:25.188434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a6242-946c-4bae-a2fb-8bc92e0f3fb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:25.189062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a6242-946c-4bae-a2fb-8bc92e0f3fb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:25.189338 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a6242-946c-4bae-a2fb-8bc92e0f3fb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:25.190788 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a6242-946c-4bae-a2fb-8bc92e0f3fb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:25.203794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a6242-946c-4bae-a2fb-8bc92e0f3fb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:25.205994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a6242-946c-4bae-a2fb-8bc92e0f3fb7 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:25.226182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a086299d-95d8-4925-9150-62cef4cadbbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1886c4475274b5481800682ea4cd98f', 'request_id': 'req-a086299d-95d8-4925-9150-62cef4cadbbc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:25.235023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1103/2195] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:26:25.236548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a086299d-95d8-4925-9150-62cef4cadbbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1886c4475274b5481800682ea4cd98f', 'request_id': 'req-a086299d-95d8-4925-9150-62cef4cadbbc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:25.246693 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a086299d-95d8-4925-9150-62cef4cadbbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1886c4475274b5481800682ea4cd98f', 'request_id': 'req-a086299d-95d8-4925-9150-62cef4cadbbc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:25.263348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a086299d-95d8-4925-9150-62cef4cadbbc admin admin] The token cache is being invalidate because project f1886c4475274b5481800682ea4cd98f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:26:25.263713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a086299d-95d8-4925-9150-62cef4cadbbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:25.265254 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a086299d-95d8-4925-9150-62cef4cadbbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:25.270286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1093/2196] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:26:25 2022] DELETE /identity/v3/projects/f1886c4475274b5481800682ea4cd98f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:26:25.272678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a6242-946c-4bae-a2fb-8bc92e0f3fb7 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:25.288435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a6242-946c-4bae-a2fb-8bc92e0f3fb7 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:25.397859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:25.398909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:25.399614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:25.400030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] PATH_INFO: `/v3/users/09e58d04403a4a22836944f3d892a3d3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:25.402242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e58d04403a4a22836944f3d892a3d3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:25.404995 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:25.484907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e58d04403a4a22836944f3d892a3d3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:25.493187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e58d04403a4a22836944f3d892a3d3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:25.494035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e58d04403a4a22836944f3d892a3d3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:25.505213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e58d04403a4a22836944f3d892a3d3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:25.538037 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e58d04403a4a22836944f3d892a3d3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:25.553873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1104/2197] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:26:25 2022] DELETE /identity/v3/users/09e58d04403a4a22836944f3d892a3d3 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:25.882664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46ab5f-c97e-4751-a664-26adc0e3f9d8 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:25.882982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46ab5f-c97e-4751-a664-26adc0e3f9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:25.883260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce46ab5f-c97e-4751-a664-26adc0e3f9d8 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:25.887531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1094/2198] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:26:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:26:25.905098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:25.909402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:25.938530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefe1c9-639b-444a-baf0-93e18cb9083b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:25.939180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefe1c9-639b-444a-baf0-93e18cb9083b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:25.940454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefe1c9-639b-444a-baf0-93e18cb9083b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:25.945077 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:25.986988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ceefe1c9-639b-444a-baf0-93e18cb9083b None None] MFA Rules not processed for user `3b16c430b870424c8d366c9a2959003e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:25.988177 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ceefe1c9-639b-444a-baf0-93e18cb9083b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.098552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31106b-bd97-4451-9757-efca39412c4f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:26.103757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:26.104425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.104770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.105858 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] PATH_INFO: `/v3/projects/d2fe812c4a7745d181fbb91a4bfafc36` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.107090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2fe812c4a7745d181fbb91a4bfafc36)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:26.110185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:26.134441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1095/2199] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:26:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:26:26.134441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefe1c9-639b-444a-baf0-93e18cb9083b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:26.136940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ceefe1c9-639b-444a-baf0-93e18cb9083b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.159247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefe1c9-639b-444a-baf0-93e18cb9083b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:26.164506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:26.165457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.165815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.166019 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] PATH_INFO: `/v3/users/e48c146c0bba4583a92d157e2eed7f99` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.167601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e48c146c0bba4583a92d157e2eed7f99)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:26.169927 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:26.192291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2fe812c4a7745d181fbb91a4bfafc36', 'request_id': 'req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.200074 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2fe812c4a7745d181fbb91a4bfafc36', 'request_id': 'req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.206763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2fe812c4a7745d181fbb91a4bfafc36', 'request_id': 'req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.220329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] The token cache is being invalidate because project d2fe812c4a7745d181fbb91a4bfafc36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:26:26.220616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.221241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.226178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e48c146c0bba4583a92d157e2eed7f99'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.226178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e48c146c0bba4583a92d157e2eed7f99'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.226425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1105/2200] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:26:25 2022] DELETE /identity/v3/projects/d2fe812c4a7745d181fbb91a4bfafc36 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:26.226425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:26.226425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.234566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e48c146c0bba4583a92d157e2eed7f99'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.244700 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e48c146c0bba4583a92d157e2eed7f99'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.261952 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e48c146c0bba4583a92d157e2eed7f99'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.268680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1096/2201] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:26:26 2022] DELETE /identity/v3/users/e48c146c0bba4583a92d157e2eed7f99 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:26.269651 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:26.294367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.301606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:26.306141 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd22f81-38aa-4651-b5f5-f5bb98104b7b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:26.365866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:26.383215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:26.384394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.384894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.386742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] PATH_INFO: `/v3/projects/1307defc25444525ba4f7d3ddd7cc4fa` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.387633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1307defc25444525ba4f7d3ddd7cc4fa)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:26.387958 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc2c66-ae61-42b4-ab6f-446d7b9ca9fd admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:26.389905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:26.394302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b7185-3c74-492f-9f6a-261f8eef914b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:26.395221 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b7185-3c74-492f-9f6a-261f8eef914b service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.395375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b7185-3c74-492f-9f6a-261f8eef914b service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.395626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b7185-3c74-492f-9f6a-261f8eef914b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.397634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a69b7185-3c74-492f-9f6a-261f8eef914b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.413011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b7185-3c74-492f-9f6a-261f8eef914b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:26.414659 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b7185-3c74-492f-9f6a-261f8eef914b service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:26.425374 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1097/2202] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:26:26.426887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8c103-b70d-4240-ac6c-c40d9eb3d855 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.427249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8c103-b70d-4240-ac6c-c40d9eb3d855 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.428083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8c103-b70d-4240-ac6c-c40d9eb3d855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.451919 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1fa8c103-b70d-4240-ac6c-c40d9eb3d855 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:26.453248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1fa8c103-b70d-4240-ac6c-c40d9eb3d855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.462353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1307defc25444525ba4f7d3ddd7cc4fa', 'request_id': 'req-9d136efc-c542-4627-bae3-8f388766d5c9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.482193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1307defc25444525ba4f7d3ddd7cc4fa', 'request_id': 'req-9d136efc-c542-4627-bae3-8f388766d5c9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.490161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1307defc25444525ba4f7d3ddd7cc4fa', 'request_id': 'req-9d136efc-c542-4627-bae3-8f388766d5c9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.506618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] The token cache is being invalidate because project 1307defc25444525ba4f7d3ddd7cc4fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:26:26.506974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.509268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.512431 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1106/2203] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:26:26 2022] DELETE /identity/v3/projects/1307defc25444525ba4f7d3ddd7cc4fa => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:26.513312 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:26.514836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.536612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1098/2204] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:26.538117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af664a-01da-43c9-9f19-d0e46feb1325 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.538284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af664a-01da-43c9-9f19-d0e46feb1325 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.538537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af664a-01da-43c9-9f19-d0e46feb1325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.565587 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-80af664a-01da-43c9-9f19-d0e46feb1325 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:26.567442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-80af664a-01da-43c9-9f19-d0e46feb1325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.576378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1099/2205] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:26.577420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3a635-b908-4757-9319-516e2a0d54d3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.577702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3a635-b908-4757-9319-516e2a0d54d3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.577975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3a635-b908-4757-9319-516e2a0d54d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.597849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6aa3a635-b908-4757-9319-516e2a0d54d3 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:26.599171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6aa3a635-b908-4757-9319-516e2a0d54d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.602721 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d136efc-c542-4627-bae3-8f388766d5c9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:26.607439 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1100/2206] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:26.607718 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:26.608816 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.609096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.609364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] PATH_INFO: `/v3/projects/9c16b9aaa5ac4e6dac613614c1a570c8` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.610703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c16b9aaa5ac4e6dac613614c1a570c8)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:26.611327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a34cb0-d0d0-4315-b884-c19f1fc081aa None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.611600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a34cb0-d0d0-4315-b884-c19f1fc081aa None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.611867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a34cb0-d0d0-4315-b884-c19f1fc081aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.612727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:26.631667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-51a34cb0-d0d0-4315-b884-c19f1fc081aa None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:26.632858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-51a34cb0-d0d0-4315-b884-c19f1fc081aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.641539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1101/2207] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:26.642267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a34cb0-d0d0-4315-b884-c19f1fc081aa None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:26.652833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a34cb0-d0d0-4315-b884-c19f1fc081aa None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:26.664442 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c16b9aaa5ac4e6dac613614c1a570c8', 'request_id': 'req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.670277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c16b9aaa5ac4e6dac613614c1a570c8', 'request_id': 'req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.680370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c16b9aaa5ac4e6dac613614c1a570c8', 'request_id': 'req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.694011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] The token cache is being invalidate because project 9c16b9aaa5ac4e6dac613614c1a570c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:26:26.694341 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.694960 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:26.696525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1107/2208] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:26:26 2022] DELETE /identity/v3/projects/9c16b9aaa5ac4e6dac613614c1a570c8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:26.697287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:26.699144 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.711381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:26.714227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a103f01-9dd1-4ee5-baa7-0de51d8be37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:26.714866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a103f01-9dd1-4ee5-baa7-0de51d8be37d admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.715142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a103f01-9dd1-4ee5-baa7-0de51d8be37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.715409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a103f01-9dd1-4ee5-baa7-0de51d8be37d admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.716055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a103f01-9dd1-4ee5-baa7-0de51d8be37d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:26:26.716363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a103f01-9dd1-4ee5-baa7-0de51d8be37d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:26.718101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a103f01-9dd1-4ee5-baa7-0de51d8be37d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:26.734052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1102/2209] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:26:26.734052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c2d4a-2b44-4f0d-800a-415884320f04 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.734052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c2d4a-2b44-4f0d-800a-415884320f04 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.734052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c2d4a-2b44-4f0d-800a-415884320f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.752549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-888c2d4a-2b44-4f0d-800a-415884320f04 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:26.753621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-888c2d4a-2b44-4f0d-800a-415884320f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.763927 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1103/2210] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:26.764911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c2d4a-2b44-4f0d-800a-415884320f04 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:26.766719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-888c2d4a-2b44-4f0d-800a-415884320f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.775288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:26.782781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c2d4a-2b44-4f0d-800a-415884320f04 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:26.782911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea195483-b7a8-4ecd-b8ef-ef49b7571d69 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:26.789776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06e322-85e2-4275-8178-98703d5a707a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:26.790474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06e322-85e2-4275-8178-98703d5a707a admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.790755 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06e322-85e2-4275-8178-98703d5a707a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.791021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06e322-85e2-4275-8178-98703d5a707a admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.791656 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511cbf3-a42b-477c-8d1a-ded5b8c87104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:26.791794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06e322-85e2-4275-8178-98703d5a707a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:26:26.791980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06e322-85e2-4275-8178-98703d5a707a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:26.792150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511cbf3-a42b-477c-8d1a-ded5b8c87104 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.792456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511cbf3-a42b-477c-8d1a-ded5b8c87104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.792726 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4511cbf3-a42b-477c-8d1a-ded5b8c87104 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.793370 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4511cbf3-a42b-477c-8d1a-ded5b8c87104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:26.793666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06e322-85e2-4275-8178-98703d5a707a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:26.795309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4511cbf3-a42b-477c-8d1a-ded5b8c87104 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:26.806046 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1108/2211] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:26:26.807314 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59ab90-a996-4fbf-8fbd-5a970e8741b6 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.807600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59ab90-a996-4fbf-8fbd-5a970e8741b6 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.807934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59ab90-a996-4fbf-8fbd-5a970e8741b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.828412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ac59ab90-a996-4fbf-8fbd-5a970e8741b6 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:26.829645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ac59ab90-a996-4fbf-8fbd-5a970e8741b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:26.839046 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1109/2212] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:26.839926 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59ab90-a996-4fbf-8fbd-5a970e8741b6 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:26.843550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1104/2213] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:26:26 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:26.844072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511cbf3-a42b-477c-8d1a-ded5b8c87104 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:26.851003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59ab90-a996-4fbf-8fbd-5a970e8741b6 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:26.906027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d0a89-48ed-4fde-aba1-51726703bbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:26.906671 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4511cbf3-a42b-477c-8d1a-ded5b8c87104 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:26.906787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d0a89-48ed-4fde-aba1-51726703bbec admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.906903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d0a89-48ed-4fde-aba1-51726703bbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.907161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d0a89-48ed-4fde-aba1-51726703bbec admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.907783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d0a89-48ed-4fde-aba1-51726703bbec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:26.909662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d0a89-48ed-4fde-aba1-51726703bbec admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:26.911080 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1eeee-796d-43b2-9a91-34fb8290a2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:26.911684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1eeee-796d-43b2-9a91-34fb8290a2fb admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:26.911962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1eeee-796d-43b2-9a91-34fb8290a2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:26.912225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1eeee-796d-43b2-9a91-34fb8290a2fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:26.912868 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1eeee-796d-43b2-9a91-34fb8290a2fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:26:26.913169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1eeee-796d-43b2-9a91-34fb8290a2fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:26.914851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1eeee-796d-43b2-9a91-34fb8290a2fb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:26.928410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1105/2214] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:26:26.930506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1eeee-796d-43b2-9a91-34fb8290a2fb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:26.942645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1eeee-796d-43b2-9a91-34fb8290a2fb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:26.943641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1110/2215] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:26:26 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:26.944388 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d0a89-48ed-4fde-aba1-51726703bbec admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.004082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4923c-d4c6-486a-b093-32b1bffc54bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.004826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4923c-d4c6-486a-b093-32b1bffc54bc admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.005098 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4923c-d4c6-486a-b093-32b1bffc54bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.005367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4923c-d4c6-486a-b093-32b1bffc54bc admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.005987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4923c-d4c6-486a-b093-32b1bffc54bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.006614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d0a89-48ed-4fde-aba1-51726703bbec admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.007914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4923c-d4c6-486a-b093-32b1bffc54bc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.011839 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04247066-e49f-4702-aabc-7444ac3137fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.012425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04247066-e49f-4702-aabc-7444ac3137fa admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.012696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04247066-e49f-4702-aabc-7444ac3137fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.012956 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04247066-e49f-4702-aabc-7444ac3137fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.013565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04247066-e49f-4702-aabc-7444ac3137fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.015465 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04247066-e49f-4702-aabc-7444ac3137fa admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.030616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1106/2216] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:26:26 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:27.031430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4923c-d4c6-486a-b093-32b1bffc54bc admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.042847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4923c-d4c6-486a-b093-32b1bffc54bc admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.050669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1111/2217] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:26:26 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:27.050812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c9520-704d-4974-902e-4f11f4209baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.050812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c9520-704d-4974-902e-4f11f4209baa admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.051032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04247066-e49f-4702-aabc-7444ac3137fa admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.051121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c9520-704d-4974-902e-4f11f4209baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.051121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c9520-704d-4974-902e-4f11f4209baa admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.051121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c9520-704d-4974-902e-4f11f4209baa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.052062 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c9520-704d-4974-902e-4f11f4209baa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.059968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:27.078102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1107/2218] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:26:27 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:27.078831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c9520-704d-4974-902e-4f11f4209baa admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.091792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c9520-704d-4974-902e-4f11f4209baa admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.122300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04247066-e49f-4702-aabc-7444ac3137fa admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.126974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d33f0-6e8f-4705-80de-2b096707239a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.127615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d33f0-6e8f-4705-80de-2b096707239a admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.127893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d33f0-6e8f-4705-80de-2b096707239a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.128181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d33f0-6e8f-4705-80de-2b096707239a admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.128829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d33f0-6e8f-4705-80de-2b096707239a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.130428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d33f0-6e8f-4705-80de-2b096707239a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.139228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:27.141349 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1112/2219] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:26:27.145104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d33f0-6e8f-4705-80de-2b096707239a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.155968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d33f0-6e8f-4705-80de-2b096707239a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.159379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b4466-ae1b-4bfa-97ae-e6ccc941bb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.160240 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b4466-ae1b-4bfa-97ae-e6ccc941bb91 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.160595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b4466-ae1b-4bfa-97ae-e6ccc941bb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.160895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b4466-ae1b-4bfa-97ae-e6ccc941bb91 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.161020 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e846b-396c-488e-8384-99657eaffcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.161518 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e846b-396c-488e-8384-99657eaffcad admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.161630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b4466-ae1b-4bfa-97ae-e6ccc941bb91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.161781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e846b-396c-488e-8384-99657eaffcad admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.162048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e846b-396c-488e-8384-99657eaffcad admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.162858 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e846b-396c-488e-8384-99657eaffcad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.163999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b4466-ae1b-4bfa-97ae-e6ccc941bb91 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.164327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e846b-396c-488e-8384-99657eaffcad admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.175095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1113/2220] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:26:27.178385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e846b-396c-488e-8384-99657eaffcad admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.190759 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e846b-396c-488e-8384-99657eaffcad admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.195977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570144a-b87d-4fd1-a741-fb4fdd03470b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.196168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1108/2221] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:26:27 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:27.200429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570144a-b87d-4fd1-a741-fb4fdd03470b admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.200429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570144a-b87d-4fd1-a741-fb4fdd03470b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.200639 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b4466-ae1b-4bfa-97ae-e6ccc941bb91 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.200713 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570144a-b87d-4fd1-a741-fb4fdd03470b admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.200713 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570144a-b87d-4fd1-a741-fb4fdd03470b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.200713 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570144a-b87d-4fd1-a741-fb4fdd03470b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.214622 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b4466-ae1b-4bfa-97ae-e6ccc941bb91 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.224443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b86bb8-3153-417a-ac05-022bfded46ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.224443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b86bb8-3153-417a-ac05-022bfded46ec admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.224443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b86bb8-3153-417a-ac05-022bfded46ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.224443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b86bb8-3153-417a-ac05-022bfded46ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.224874 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b86bb8-3153-417a-ac05-022bfded46ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.225158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b86bb8-3153-417a-ac05-022bfded46ec admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.226086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1114/2222] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:26:27.227522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570144a-b87d-4fd1-a741-fb4fdd03470b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.242203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1109/2223] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:26:27.243150 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b86bb8-3153-417a-ac05-022bfded46ec admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.244408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570144a-b87d-4fd1-a741-fb4fdd03470b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.252438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6630e2-c1b0-444c-a77c-07cee5fb777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.253505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6630e2-c1b0-444c-a77c-07cee5fb777f admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.253784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6630e2-c1b0-444c-a77c-07cee5fb777f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.254101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6630e2-c1b0-444c-a77c-07cee5fb777f admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.254398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b86bb8-3153-417a-ac05-022bfded46ec admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.254794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6630e2-c1b0-444c-a77c-07cee5fb777f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.256935 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6630e2-c1b0-444c-a77c-07cee5fb777f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.259883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7617bd9-3840-4384-88b5-59527aa55815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.260587 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7617bd9-3840-4384-88b5-59527aa55815 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.260871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7617bd9-3840-4384-88b5-59527aa55815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.261173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7617bd9-3840-4384-88b5-59527aa55815 admin admin] PATH_INFO: `/v3/projects/9bf4e20a59f147afb4f036fc8d04adb2/users/c3acafe7ed414c87a5ca235b96047aca/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.267617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1115/2224] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:26:27.271536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6630e2-c1b0-444c-a77c-07cee5fb777f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.283593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6630e2-c1b0-444c-a77c-07cee5fb777f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.283791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7617bd9-3840-4384-88b5-59527aa55815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf4e20a59f147afb4f036fc8d04adb2, user_id=c3acafe7ed414c87a5ca235b96047aca, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.285847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7617bd9-3840-4384-88b5-59527aa55815 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.288175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef86ae-fbe9-46cc-9dcc-30e0508c6728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.288905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef86ae-fbe9-46cc-9dcc-30e0508c6728 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.289249 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef86ae-fbe9-46cc-9dcc-30e0508c6728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.289583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef86ae-fbe9-46cc-9dcc-30e0508c6728 admin admin] PATH_INFO: `/v3/projects/5560ba31a85e4e6a9d04dbc5bd592c17/users/b7b1959adde54f99bd2234b3a12bdd0a/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.307522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef86ae-fbe9-46cc-9dcc-30e0508c6728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5560ba31a85e4e6a9d04dbc5bd592c17, user_id=b7b1959adde54f99bd2234b3a12bdd0a, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.312558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef86ae-fbe9-46cc-9dcc-30e0508c6728 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.322586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1110/2225] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:26:27 2022] PUT /identity/v3/projects/9bf4e20a59f147afb4f036fc8d04adb2/users/c3acafe7ed414c87a5ca235b96047aca/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:27.323474 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7617bd9-3840-4384-88b5-59527aa55815 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.335420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1116/2226] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:26:27 2022] PUT /identity/v3/projects/5560ba31a85e4e6a9d04dbc5bd592c17/users/b7b1959adde54f99bd2234b3a12bdd0a/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:27.337923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6ab99-3eee-415d-90fa-0d673c477c81 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.338850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6ab99-3eee-415d-90fa-0d673c477c81 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.339340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6ab99-3eee-415d-90fa-0d673c477c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.381420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-16d6ab99-3eee-415d-90fa-0d673c477c81 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:27.381420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-16d6ab99-3eee-415d-90fa-0d673c477c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:27.418593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7617bd9-3840-4384-88b5-59527aa55815 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.447181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1117/2227] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:27.451587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83875668-7f28-45c4-9c8f-c5bd73f877b2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.451875 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83875668-7f28-45c4-9c8f-c5bd73f877b2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.452152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83875668-7f28-45c4-9c8f-c5bd73f877b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.476882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd30a27-d0ba-487b-930b-3567d24df2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.477478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd30a27-d0ba-487b-930b-3567d24df2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.477751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd30a27-d0ba-487b-930b-3567d24df2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.478019 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd30a27-d0ba-487b-930b-3567d24df2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.478681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd30a27-d0ba-487b-930b-3567d24df2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.479222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-83875668-7f28-45c4-9c8f-c5bd73f877b2 None None] MFA Rules not processed for user `b7b1959adde54f99bd2234b3a12bdd0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:27.481120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd30a27-d0ba-487b-930b-3567d24df2d3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.481239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-83875668-7f28-45c4-9c8f-c5bd73f877b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:27.491690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1111/2228] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:26:27.492504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd30a27-d0ba-487b-930b-3567d24df2d3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.496503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ecd30a27-d0ba-487b-930b-3567d24df2d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:27.530184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:27.544251 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:27.556822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd30a27-d0ba-487b-930b-3567d24df2d3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.561401 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a61de1-5cad-4f46-8c32-b6f7820efdc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.562026 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a61de1-5cad-4f46-8c32-b6f7820efdc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.562343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a61de1-5cad-4f46-8c32-b6f7820efdc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.562703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a61de1-5cad-4f46-8c32-b6f7820efdc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.572292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1118/2229] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:27.572292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83875668-7f28-45c4-9c8f-c5bd73f877b2 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.577830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a61de1-5cad-4f46-8c32-b6f7820efdc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.579541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a61de1-5cad-4f46-8c32-b6f7820efdc5 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.582465 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83875668-7f28-45c4-9c8f-c5bd73f877b2 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.587437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8cf55-cc57-4eb3-866f-e135300a00f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.588032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8cf55-cc57-4eb3-866f-e135300a00f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.588341 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8cf55-cc57-4eb3-866f-e135300a00f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.588614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8cf55-cc57-4eb3-866f-e135300a00f6 admin admin] PATH_INFO: `/v3/projects/ef13103d3d8b43e1882d4820c8d9247b/users/0b02c2ba9a0a48c294a11b42cbfe28e3/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.589869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1112/2230] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:26:27.590757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a61de1-5cad-4f46-8c32-b6f7820efdc5 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.592095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-64a61de1-5cad-4f46-8c32-b6f7820efdc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:27.608255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8cf55-cc57-4eb3-866f-e135300a00f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef13103d3d8b43e1882d4820c8d9247b, user_id=0b02c2ba9a0a48c294a11b42cbfe28e3, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.611315 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8cf55-cc57-4eb3-866f-e135300a00f6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.630937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1119/2231] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:26:27 2022] PUT /identity/v3/projects/ef13103d3d8b43e1882d4820c8d9247b/users/0b02c2ba9a0a48c294a11b42cbfe28e3/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:27.631998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63a3dd-42c3-4fb7-bf83-d42e97639c72 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.632585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63a3dd-42c3-4fb7-bf83-d42e97639c72 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.632696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63a3dd-42c3-4fb7-bf83-d42e97639c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.657664 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5c63a3dd-42c3-4fb7-bf83-d42e97639c72 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:27.659082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5c63a3dd-42c3-4fb7-bf83-d42e97639c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:27.662864 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a61de1-5cad-4f46-8c32-b6f7820efdc5 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.738113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1120/2232] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:26:27.742904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90120332-af8d-4042-9932-c299f947c6b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.743526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90120332-af8d-4042-9932-c299f947c6b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.743805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90120332-af8d-4042-9932-c299f947c6b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.744077 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90120332-af8d-4042-9932-c299f947c6b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.749165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63a3dd-42c3-4fb7-bf83-d42e97639c72 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:27.751544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5c63a3dd-42c3-4fb7-bf83-d42e97639c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:27.821030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90120332-af8d-4042-9932-c299f947c6b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.822704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90120332-af8d-4042-9932-c299f947c6b5 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.840918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63a3dd-42c3-4fb7-bf83-d42e97639c72 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:27.841623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1113/2233] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:26:27.845772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83590a4d-fa34-4c51-8a45-025008084ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:27.846589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83590a4d-fa34-4c51-8a45-025008084ffe service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:27.846934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83590a4d-fa34-4c51-8a45-025008084ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:27.847329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83590a4d-fa34-4c51-8a45-025008084ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:27.860563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83590a4d-fa34-4c51-8a45-025008084ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:27.862265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83590a4d-fa34-4c51-8a45-025008084ffe service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:27.876479 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1121/2234] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:26:35.335820 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9373b-d2c9-4f24-a870-b7edfb1910ce None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:35.336332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9373b-d2c9-4f24-a870-b7edfb1910ce None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:35.336429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9373b-d2c9-4f24-a870-b7edfb1910ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:35.372465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-aba9373b-d2c9-4f24-a870-b7edfb1910ce None None] MFA Rules not processed for user `c3acafe7ed414c87a5ca235b96047aca`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:35.374042 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-aba9373b-d2c9-4f24-a870-b7edfb1910ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:35.425493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:35.448942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1114/2235] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:35.453400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83590a4d-fa34-4c51-8a45-025008084ffe service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:35.464539 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83590a4d-fa34-4c51-8a45-025008084ffe service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:35.471906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd2839-e386-4cdf-954e-e81bc5af84d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:35.473286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd2839-e386-4cdf-954e-e81bc5af84d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:35.473286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd2839-e386-4cdf-954e-e81bc5af84d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:35.473601 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd2839-e386-4cdf-954e-e81bc5af84d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:35.474316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd2839-e386-4cdf-954e-e81bc5af84d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:35.476828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd2839-e386-4cdf-954e-e81bc5af84d9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:35.523048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1122/2236] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:26:35 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:35.527627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9373b-d2c9-4f24-a870-b7edfb1910ce None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:35.538860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9373b-d2c9-4f24-a870-b7edfb1910ce None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:35.620125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d6f3e-119f-48eb-8bd9-25069a38db9a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:35.620391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d6f3e-119f-48eb-8bd9-25069a38db9a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:35.620661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d6f3e-119f-48eb-8bd9-25069a38db9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:35.629717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643e1f4-d5e3-4e5c-ab25-cfddd27164c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:35.630592 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643e1f4-d5e3-4e5c-ab25-cfddd27164c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:35.630871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643e1f4-d5e3-4e5c-ab25-cfddd27164c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:35.631164 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643e1f4-d5e3-4e5c-ab25-cfddd27164c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:35.631820 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643e1f4-d5e3-4e5c-ab25-cfddd27164c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:35.633702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643e1f4-d5e3-4e5c-ab25-cfddd27164c6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:35.665962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1115/2237] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:26:35 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:35.675136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643e1f4-d5e3-4e5c-ab25-cfddd27164c6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:35.681930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643e1f4-d5e3-4e5c-ab25-cfddd27164c6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:35.685135 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a56d6f3e-119f-48eb-8bd9-25069a38db9a None None] MFA Rules not processed for user `0b02c2ba9a0a48c294a11b42cbfe28e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:35.686413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a56d6f3e-119f-48eb-8bd9-25069a38db9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:35.687496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6b1c0-a4b2-4eda-9cc5-7b59b4466b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:35.688229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6b1c0-a4b2-4eda-9cc5-7b59b4466b2d admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:35.688648 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6b1c0-a4b2-4eda-9cc5-7b59b4466b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:35.689015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc6b1c0-a4b2-4eda-9cc5-7b59b4466b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:35.689855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6b1c0-a4b2-4eda-9cc5-7b59b4466b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:35.692105 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc6b1c0-a4b2-4eda-9cc5-7b59b4466b2d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:35.705234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1116/2238] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:26:35.711179 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6b1c0-a4b2-4eda-9cc5-7b59b4466b2d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:35.724257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc6b1c0-a4b2-4eda-9cc5-7b59b4466b2d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:35.731607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65629e-1d17-45b8-a70e-5f5cc3e715ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:35.732362 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65629e-1d17-45b8-a70e-5f5cc3e715ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:35.732718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65629e-1d17-45b8-a70e-5f5cc3e715ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:35.733120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65629e-1d17-45b8-a70e-5f5cc3e715ce admin admin] PATH_INFO: `/v3/projects/693365fee2754e3e8ac69052733795a3/users/8d0791cc553a41deadf1dd0c0195482b/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:35.740907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:35.761858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65629e-1d17-45b8-a70e-5f5cc3e715ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=693365fee2754e3e8ac69052733795a3, user_id=8d0791cc553a41deadf1dd0c0195482b, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:35.766633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65629e-1d17-45b8-a70e-5f5cc3e715ce admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:35.788780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1117/2239] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:26:35 2022] PUT /identity/v3/projects/693365fee2754e3e8ac69052733795a3/users/8d0791cc553a41deadf1dd0c0195482b/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:35.789836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65629e-1d17-45b8-a70e-5f5cc3e715ce admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:35.814978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1123/2240] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:26:35.823609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d6f3e-119f-48eb-8bd9-25069a38db9a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:35.824839 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a56d6f3e-119f-48eb-8bd9-25069a38db9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:35.875115 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65629e-1d17-45b8-a70e-5f5cc3e715ce admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:35.885267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc53eff-d733-4923-821e-6f270e659c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:35.886666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc53eff-d733-4923-821e-6f270e659c72 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:35.886944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc53eff-d733-4923-821e-6f270e659c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:35.887214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc53eff-d733-4923-821e-6f270e659c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:35.887998 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc53eff-d733-4923-821e-6f270e659c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:35.889774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc53eff-d733-4923-821e-6f270e659c72 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:35.902128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1118/2241] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:26:35.906355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc53eff-d733-4923-821e-6f270e659c72 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:35.918402 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc53eff-d733-4923-821e-6f270e659c72 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:35.948893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff9c79-bb67-49f4-800c-793ffb93cb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:35.949569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff9c79-bb67-49f4-800c-793ffb93cb29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:35.949844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff9c79-bb67-49f4-800c-793ffb93cb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:35.950146 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff9c79-bb67-49f4-800c-793ffb93cb29 admin admin] PATH_INFO: `/v3/projects/693365fee2754e3e8ac69052733795a3/users/8d0791cc553a41deadf1dd0c0195482b/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:35.952383 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff9c79-bb67-49f4-800c-793ffb93cb29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=693365fee2754e3e8ac69052733795a3, user_id=8d0791cc553a41deadf1dd0c0195482b, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:35.955772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff9c79-bb67-49f4-800c-793ffb93cb29 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:35.968686 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1119/2242] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:26:35 2022] PUT /identity/v3/projects/693365fee2754e3e8ac69052733795a3/users/8d0791cc553a41deadf1dd0c0195482b/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:26:36.023432 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d6f3e-119f-48eb-8bd9-25069a38db9a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:36.200382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4cf55-523b-4bb5-9685-b6ddfcaf6536 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:36.201119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4cf55-523b-4bb5-9685-b6ddfcaf6536 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:36.201420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4cf55-523b-4bb5-9685-b6ddfcaf6536 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:36.201686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4cf55-523b-4bb5-9685-b6ddfcaf6536 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:36.314027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4cf55-523b-4bb5-9685-b6ddfcaf6536 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:36.316411 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4cf55-523b-4bb5-9685-b6ddfcaf6536 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:36.342678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:36.348155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1124/2243] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:26:38.769891 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c67ac3-d53d-4d58-b032-bbcbf72f5ae6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:38.770477 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c67ac3-d53d-4d58-b032-bbcbf72f5ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:38.770477 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c67ac3-d53d-4d58-b032-bbcbf72f5ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:38.807206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e8c67ac3-d53d-4d58-b032-bbcbf72f5ae6 None None] MFA Rules not processed for user `0b02c2ba9a0a48c294a11b42cbfe28e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:38.810057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e8c67ac3-d53d-4d58-b032-bbcbf72f5ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:38.832761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1120/2244] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:39.551953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4cf55-523b-4bb5-9685-b6ddfcaf6536 service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:39.553472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3fc4cf55-523b-4bb5-9685-b6ddfcaf6536 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:39.657144 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4cf55-523b-4bb5-9685-b6ddfcaf6536 service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:39.664099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:39.664774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:39.665084 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:39.665312 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] PATH_INFO: `/v3/users/7417a079de114575840c6d3f1c9716e0` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:39.667242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7417a079de114575840c6d3f1c9716e0)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:39.672949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:39.792803 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7417a079de114575840c6d3f1c9716e0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:39.804746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7417a079de114575840c6d3f1c9716e0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:39.805747 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7417a079de114575840c6d3f1c9716e0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:39.817531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7417a079de114575840c6d3f1c9716e0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:39.836146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7417a079de114575840c6d3f1c9716e0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:39.851492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1125/2245] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:26:39 2022] DELETE /identity/v3/users/7417a079de114575840c6d3f1c9716e0 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:39.852537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db3f4c-3097-4452-8010-e9cb8249f874 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:39.853061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db3f4c-3097-4452-8010-e9cb8249f874 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:39.853061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db3f4c-3097-4452-8010-e9cb8249f874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:39.894442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-93db3f4c-3097-4452-8010-e9cb8249f874 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:39.898051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-93db3f4c-3097-4452-8010-e9cb8249f874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:39.918586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:40.104566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1121/2246] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:26:40.113801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:40.117389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:40.326242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d16f5d-6509-4514-8b8d-7688b5c010c9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:40.344294 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b28877-9bac-47c6-ad89-b84d279c62be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:40.345119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b28877-9bac-47c6-ad89-b84d279c62be service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:40.345533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b28877-9bac-47c6-ad89-b84d279c62be service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:40.345813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b28877-9bac-47c6-ad89-b84d279c62be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:40.390650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b28877-9bac-47c6-ad89-b84d279c62be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:40.392288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b28877-9bac-47c6-ad89-b84d279c62be service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:40.415488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1126/2247] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:26:41.029881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db3f4c-3097-4452-8010-e9cb8249f874 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:41.031441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-93db3f4c-3097-4452-8010-e9cb8249f874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:41.070033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db3f4c-3097-4452-8010-e9cb8249f874 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:41.081142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:41.086281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:41.086658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:41.087000 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] PATH_INFO: `/v3/projects/1fcdacccb639490ba9986dace34a60cc` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:41.091560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fcdacccb639490ba9986dace34a60cc)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:41.096934 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:41.251347 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fcdacccb639490ba9986dace34a60cc', 'request_id': 'req-f67e8b3e-b113-404e-8007-224a64a2be41'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:41.279470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fcdacccb639490ba9986dace34a60cc', 'request_id': 'req-f67e8b3e-b113-404e-8007-224a64a2be41'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:41.304014 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fcdacccb639490ba9986dace34a60cc', 'request_id': 'req-f67e8b3e-b113-404e-8007-224a64a2be41'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:41.368718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] The token cache is being invalidate because project 1fcdacccb639490ba9986dace34a60cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:26:41.369289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:41.370474 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:41.374821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1122/2248] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:26:41 2022] DELETE /identity/v3/projects/1fcdacccb639490ba9986dace34a60cc => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:41.379275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b28877-9bac-47c6-ad89-b84d279c62be service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:41.411191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b28877-9bac-47c6-ad89-b84d279c62be service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:41.467988 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:41.573648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d15b9-3273-4cbd-8df3-35be44708ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:41.574692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d15b9-3273-4cbd-8df3-35be44708ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:41.575129 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d15b9-3273-4cbd-8df3-35be44708ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:41.575537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d15b9-3273-4cbd-8df3-35be44708ebe admin admin] PATH_INFO: `/v3/users/d732f56ab58848b4bfb57bf5e0f50a28` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:41.576874 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d15b9-3273-4cbd-8df3-35be44708ebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d732f56ab58848b4bfb57bf5e0f50a28)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:41.579535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d15b9-3273-4cbd-8df3-35be44708ebe admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:41.693815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-219d15b9-3273-4cbd-8df3-35be44708ebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd732f56ab58848b4bfb57bf5e0f50a28'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:41.706179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-219d15b9-3273-4cbd-8df3-35be44708ebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd732f56ab58848b4bfb57bf5e0f50a28'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:41.707397 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-219d15b9-3273-4cbd-8df3-35be44708ebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd732f56ab58848b4bfb57bf5e0f50a28'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:41.746666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-219d15b9-3273-4cbd-8df3-35be44708ebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd732f56ab58848b4bfb57bf5e0f50a28'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:41.779735 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-219d15b9-3273-4cbd-8df3-35be44708ebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd732f56ab58848b4bfb57bf5e0f50a28'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:41.793881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1127/2249] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:26:41 2022] DELETE /identity/v3/users/d732f56ab58848b4bfb57bf5e0f50a28 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:42.433147 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:42.435506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:42.481170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67e8b3e-b113-404e-8007-224a64a2be41 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:42.661737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d02288-82f4-463a-b97f-8bbd0a62a93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:42.663514 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d02288-82f4-463a-b97f-8bbd0a62a93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:42.663816 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d02288-82f4-463a-b97f-8bbd0a62a93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:42.664064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d02288-82f4-463a-b97f-8bbd0a62a93a admin admin] PATH_INFO: `/v3/projects/ac96be16ecf7477d86d75724922237f8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:42.670339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d02288-82f4-463a-b97f-8bbd0a62a93a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac96be16ecf7477d86d75724922237f8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:42.675442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d02288-82f4-463a-b97f-8bbd0a62a93a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:42.819017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:42.847749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e5d02288-82f4-463a-b97f-8bbd0a62a93a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac96be16ecf7477d86d75724922237f8', 'request_id': 'req-e5d02288-82f4-463a-b97f-8bbd0a62a93a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:42.863984 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e5d02288-82f4-463a-b97f-8bbd0a62a93a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac96be16ecf7477d86d75724922237f8', 'request_id': 'req-e5d02288-82f4-463a-b97f-8bbd0a62a93a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:42.877899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e5d02288-82f4-463a-b97f-8bbd0a62a93a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac96be16ecf7477d86d75724922237f8', 'request_id': 'req-e5d02288-82f4-463a-b97f-8bbd0a62a93a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:42.928546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e5d02288-82f4-463a-b97f-8bbd0a62a93a admin admin] The token cache is being invalidate because project ac96be16ecf7477d86d75724922237f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:26:42.929991 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e5d02288-82f4-463a-b97f-8bbd0a62a93a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:42.934628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e5d02288-82f4-463a-b97f-8bbd0a62a93a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:26:42.946954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1123/2250] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:26:42 2022] DELETE /identity/v3/projects/ac96be16ecf7477d86d75724922237f8 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:42.947932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa3168-2875-4295-9305-ac20ef7cbc13 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:42.948919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa3168-2875-4295-9305-ac20ef7cbc13 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:42.949025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa3168-2875-4295-9305-ac20ef7cbc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:43.045694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-51aa3168-2875-4295-9305-ac20ef7cbc13 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:43.052101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-51aa3168-2875-4295-9305-ac20ef7cbc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:43.343583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fca059-264a-480b-ad82-e43ae1ea53d7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:43.343924 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1128/2251] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:43.344035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fca059-264a-480b-ad82-e43ae1ea53d7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:43.344182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fca059-264a-480b-ad82-e43ae1ea53d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:43.433668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-96fca059-264a-480b-ad82-e43ae1ea53d7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:43.435041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-96fca059-264a-480b-ad82-e43ae1ea53d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:43.458912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1124/2252] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:43.465144 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa3168-2875-4295-9305-ac20ef7cbc13 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:43.481598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa3168-2875-4295-9305-ac20ef7cbc13 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:43.494970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d1d68-b2b3-40b3-b7b5-903fb712c76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:43.495602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d1d68-b2b3-40b3-b7b5-903fb712c76f admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:43.495882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d1d68-b2b3-40b3-b7b5-903fb712c76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:43.496154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d1d68-b2b3-40b3-b7b5-903fb712c76f admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:43.496823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d1d68-b2b3-40b3-b7b5-903fb712c76f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:26:43.498757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d1d68-b2b3-40b3-b7b5-903fb712c76f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:43.509339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d1d68-b2b3-40b3-b7b5-903fb712c76f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:43.528197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1129/2253] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:26:43.533233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fca059-264a-480b-ad82-e43ae1ea53d7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:43.545637 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fca059-264a-480b-ad82-e43ae1ea53d7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:43.557461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fc293-7d77-491c-87aa-b64b1d4cfc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:43.558419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fc293-7d77-491c-87aa-b64b1d4cfc86 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:43.558809 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fc293-7d77-491c-87aa-b64b1d4cfc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:43.559178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fc293-7d77-491c-87aa-b64b1d4cfc86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:43.560656 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fc293-7d77-491c-87aa-b64b1d4cfc86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:43.562845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fc293-7d77-491c-87aa-b64b1d4cfc86 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:43.605714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1125/2254] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:26:43 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:43.609840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d1d68-b2b3-40b3-b7b5-903fb712c76f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:43.627922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d1d68-b2b3-40b3-b7b5-903fb712c76f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:43.685780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:43.731267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b185618-a809-487e-886c-ffdaa4e537dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:43.732113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b185618-a809-487e-886c-ffdaa4e537dd admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:43.732397 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b185618-a809-487e-886c-ffdaa4e537dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:43.732672 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b185618-a809-487e-886c-ffdaa4e537dd admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:43.733324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b185618-a809-487e-886c-ffdaa4e537dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:43.735344 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b185618-a809-487e-886c-ffdaa4e537dd admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:43.766657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1130/2255] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:26:43 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:43.771345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fc293-7d77-491c-87aa-b64b1d4cfc86 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:43.786363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fc293-7d77-491c-87aa-b64b1d4cfc86 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:43.793832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33cd12-b64f-473f-a01c-9c9ccf3e22e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:43.794843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33cd12-b64f-473f-a01c-9c9ccf3e22e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:43.795308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33cd12-b64f-473f-a01c-9c9ccf3e22e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:43.795830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33cd12-b64f-473f-a01c-9c9ccf3e22e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:43.796926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33cd12-b64f-473f-a01c-9c9ccf3e22e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:43.798798 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33cd12-b64f-473f-a01c-9c9ccf3e22e6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:43.816101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1126/2256] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:26:43.818013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b185618-a809-487e-886c-ffdaa4e537dd admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:43.830250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b185618-a809-487e-886c-ffdaa4e537dd admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:43.836575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cab63-184b-482e-b46f-4d54b496f85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:43.837512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cab63-184b-482e-b46f-4d54b496f85e admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:43.838168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cab63-184b-482e-b46f-4d54b496f85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:43.838631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cab63-184b-482e-b46f-4d54b496f85e admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:43.839440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cab63-184b-482e-b46f-4d54b496f85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:43.841147 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cab63-184b-482e-b46f-4d54b496f85e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:43.858001 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1131/2257] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:26:43.860090 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33cd12-b64f-473f-a01c-9c9ccf3e22e6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:43.874384 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33cd12-b64f-473f-a01c-9c9ccf3e22e6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:43.880770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9252ba-339d-4b17-b24c-d44716f149bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:43.881609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9252ba-339d-4b17-b24c-d44716f149bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:43.882113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9252ba-339d-4b17-b24c-d44716f149bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:43.882427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9252ba-339d-4b17-b24c-d44716f149bb admin admin] PATH_INFO: `/v3/projects/879e23eef9ae49a1950fb25401173647/users/18ee56757cf3470d88afea4ca0540bd1/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:43.902375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9252ba-339d-4b17-b24c-d44716f149bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=879e23eef9ae49a1950fb25401173647, user_id=18ee56757cf3470d88afea4ca0540bd1, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:43.905714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9252ba-339d-4b17-b24c-d44716f149bb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:43.935306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1127/2258] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:26:43 2022] PUT /identity/v3/projects/879e23eef9ae49a1950fb25401173647/users/18ee56757cf3470d88afea4ca0540bd1/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:43.938274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb71d7-2884-4f7a-b4e0-374b60624485 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:43.938850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb71d7-2884-4f7a-b4e0-374b60624485 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:43.939283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb71d7-2884-4f7a-b4e0-374b60624485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:43.994178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-19fb71d7-2884-4f7a-b4e0-374b60624485 None None] MFA Rules not processed for user `18ee56757cf3470d88afea4ca0540bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:43.999616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-19fb71d7-2884-4f7a-b4e0-374b60624485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:44.162641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1132/2259] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:44.163966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9252ba-339d-4b17-b24c-d44716f149bb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:44.331094 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9252ba-339d-4b17-b24c-d44716f149bb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:44.339629 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b2d49-ab95-4547-b5f9-a0d23c223111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:44.340275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b2d49-ab95-4547-b5f9-a0d23c223111 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:44.340532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b2d49-ab95-4547-b5f9-a0d23c223111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:44.340791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b2d49-ab95-4547-b5f9-a0d23c223111 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:44.341434 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b2d49-ab95-4547-b5f9-a0d23c223111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:44.351323 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b2d49-ab95-4547-b5f9-a0d23c223111 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:44.404127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1128/2260] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:26:44 2022] POST /identity/v3/projects => generated 381 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:44.408104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb71d7-2884-4f7a-b4e0-374b60624485 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:44.424251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb71d7-2884-4f7a-b4e0-374b60624485 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:44.570370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea726b61-a548-454c-ae5f-e99b1227ab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:44.571162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea726b61-a548-454c-ae5f-e99b1227ab25 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:44.571551 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea726b61-a548-454c-ae5f-e99b1227ab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:44.571922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea726b61-a548-454c-ae5f-e99b1227ab25 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:44.572657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea726b61-a548-454c-ae5f-e99b1227ab25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:44.578050 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea726b61-a548-454c-ae5f-e99b1227ab25 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:44.623331 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1133/2261] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:26:44 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:26:44.625235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b2d49-ab95-4547-b5f9-a0d23c223111 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:44.649758 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b2d49-ab95-4547-b5f9-a0d23c223111 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:44.672231 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d74065-2600-4f70-a605-8ce2d510ed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:44.672861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d74065-2600-4f70-a605-8ce2d510ed95 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:44.675464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d74065-2600-4f70-a605-8ce2d510ed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:44.675926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d74065-2600-4f70-a605-8ce2d510ed95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:44.679040 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d74065-2600-4f70-a605-8ce2d510ed95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:44.680895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d74065-2600-4f70-a605-8ce2d510ed95 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:44.710646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1129/2262] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:26:44.715520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea726b61-a548-454c-ae5f-e99b1227ab25 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:44.731239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea726b61-a548-454c-ae5f-e99b1227ab25 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:44.750352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e97fe1-deec-40db-af0b-e3f9012039a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:44.750352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e97fe1-deec-40db-af0b-e3f9012039a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:44.750352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e97fe1-deec-40db-af0b-e3f9012039a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:44.750352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e97fe1-deec-40db-af0b-e3f9012039a4 admin admin] PATH_INFO: `/v3/projects/141d8e7566f444a6857cf96560b8b68b/users/5149f40aabb34f9aabdd632013fe6ed8/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:44.752728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad958-8221-4073-b7b7-65d48513efee None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:44.753248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad958-8221-4073-b7b7-65d48513efee None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:44.753661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad958-8221-4073-b7b7-65d48513efee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:44.778039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e97fe1-deec-40db-af0b-e3f9012039a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=141d8e7566f444a6857cf96560b8b68b, user_id=5149f40aabb34f9aabdd632013fe6ed8, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:44.780913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e97fe1-deec-40db-af0b-e3f9012039a4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:44.826446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1134/2263] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:26:44 2022] PUT /identity/v3/projects/141d8e7566f444a6857cf96560b8b68b/users/5149f40aabb34f9aabdd632013fe6ed8/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:26:44.833697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e97fe1-deec-40db-af0b-e3f9012039a4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:44.839429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:44.864772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bd5ad958-8221-4073-b7b7-65d48513efee None None] MFA Rules not processed for user `8d0791cc553a41deadf1dd0c0195482b`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:44.866777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bd5ad958-8221-4073-b7b7-65d48513efee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:44.991193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e97fe1-deec-40db-af0b-e3f9012039a4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:45.000325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:45.002538 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4c7d9-aef4-4939-861c-587cffb7a2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:45.003209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4c7d9-aef4-4939-861c-587cffb7a2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:45.003490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4c7d9-aef4-4939-861c-587cffb7a2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:45.003757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4c7d9-aef4-4939-861c-587cffb7a2d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:45.004412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4c7d9-aef4-4939-861c-587cffb7a2d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:45.009409 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4c7d9-aef4-4939-861c-587cffb7a2d1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:45.039508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1135/2264] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:26:45.041495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4c7d9-aef4-4939-861c-587cffb7a2d1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:45.070419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4c7d9-aef4-4939-861c-587cffb7a2d1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:45.099758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db6e70-fe92-4642-90f7-ba0f782e2ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:45.102059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db6e70-fe92-4642-90f7-ba0f782e2ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:45.111098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db6e70-fe92-4642-90f7-ba0f782e2ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:45.111518 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db6e70-fe92-4642-90f7-ba0f782e2ab1 admin admin] PATH_INFO: `/v3/projects/141d8e7566f444a6857cf96560b8b68b/users/5149f40aabb34f9aabdd632013fe6ed8/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:45.117372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db6e70-fe92-4642-90f7-ba0f782e2ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=141d8e7566f444a6857cf96560b8b68b, user_id=5149f40aabb34f9aabdd632013fe6ed8, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:45.124901 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db6e70-fe92-4642-90f7-ba0f782e2ab1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:45.139544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1130/2265] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:45.150577 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1136/2266] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:26:45 2022] PUT /identity/v3/projects/141d8e7566f444a6857cf96560b8b68b/users/5149f40aabb34f9aabdd632013fe6ed8/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:26:45.153905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ad958-8221-4073-b7b7-65d48513efee None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:45.159628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bd5ad958-8221-4073-b7b7-65d48513efee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:45.165954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f683df5-719d-46a3-9c5c-c947fa12b270 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:45.168426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f683df5-719d-46a3-9c5c-c947fa12b270 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:45.170631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f683df5-719d-46a3-9c5c-c947fa12b270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:45.237817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1f683df5-719d-46a3-9c5c-c947fa12b270 None None] MFA Rules not processed for user `5149f40aabb34f9aabdd632013fe6ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:45.239108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1f683df5-719d-46a3-9c5c-c947fa12b270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:45.294287 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ad958-8221-4073-b7b7-65d48513efee None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:45.391401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1137/2267] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:45.398912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec989d41-6173-4a46-8e96-7c05ab0aabd0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc89c1a7e684612a73ec9fef6bda780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:45.399549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec989d41-6173-4a46-8e96-7c05ab0aabd0 service glance] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:45.399830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec989d41-6173-4a46-8e96-7c05ab0aabd0 service glance] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:45.400167 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec989d41-6173-4a46-8e96-7c05ab0aabd0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:45.402902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ec989d41-6173-4a46-8e96-7c05ab0aabd0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:45.427523 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f683df5-719d-46a3-9c5c-c947fa12b270 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:45.429037 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1f683df5-719d-46a3-9c5c-c947fa12b270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:45.484147 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec989d41-6173-4a46-8e96-7c05ab0aabd0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:45.489359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec989d41-6173-4a46-8e96-7c05ab0aabd0 service glance] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:45.503236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1131/2268] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:26:45.580058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f683df5-719d-46a3-9c5c-c947fa12b270 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:45.584576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784de44-2551-4b96-8953-792af6657f90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:45.585317 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784de44-2551-4b96-8953-792af6657f90 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:45.585659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784de44-2551-4b96-8953-792af6657f90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:45.586017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784de44-2551-4b96-8953-792af6657f90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:45.608692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784de44-2551-4b96-8953-792af6657f90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:45.610290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784de44-2551-4b96-8953-792af6657f90 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:45.653075 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1138/2269] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:26:45.857102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec989d41-6173-4a46-8e96-7c05ab0aabd0 service glance] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:45.884658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec989d41-6173-4a46-8e96-7c05ab0aabd0 service glance] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:45.922174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba02fca-922c-4290-a80c-fcfb35746954 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:45.922704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba02fca-922c-4290-a80c-fcfb35746954 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:45.922987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba02fca-922c-4290-a80c-fcfb35746954 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:45.923263 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba02fca-922c-4290-a80c-fcfb35746954 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:45.963247 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce874cde-5105-48e6-a44e-e399aa85e3f9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:45.963754 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce874cde-5105-48e6-a44e-e399aa85e3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:45.964135 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce874cde-5105-48e6-a44e-e399aa85e3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:46.015838 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ce874cde-5105-48e6-a44e-e399aa85e3f9 None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:46.018340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ce874cde-5105-48e6-a44e-e399aa85e3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:46.033892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:46.037926 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:46.093891 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba02fca-922c-4290-a80c-fcfb35746954 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:46.095858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba02fca-922c-4290-a80c-fcfb35746954 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:46.116592 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1132/2270] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:26:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:26:46.146681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1139/2271] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:26:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:26:46.228093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba02fca-922c-4290-a80c-fcfb35746954 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:26:46.230336 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2ba02fca-922c-4290-a80c-fcfb35746954 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:46.452388 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba02fca-922c-4290-a80c-fcfb35746954 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:26:46.460023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae49d7a-5ed4-4963-826a-8fe3b2cee407 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:26:46.460891 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae49d7a-5ed4-4963-826a-8fe3b2cee407 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:46.461321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae49d7a-5ed4-4963-826a-8fe3b2cee407 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:46.461609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae49d7a-5ed4-4963-826a-8fe3b2cee407 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:46.476223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae49d7a-5ed4-4963-826a-8fe3b2cee407 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:26:46.478186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae49d7a-5ed4-4963-826a-8fe3b2cee407 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:26:46.488742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1133/2272] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:26:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:26:56.138526 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=19 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:56.486397 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=43 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:26:57.444914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ce757-6181-438d-a1da-4950be98bf12 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:26:57.445394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ce757-6181-438d-a1da-4950be98bf12 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:26:57.445394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ce757-6181-438d-a1da-4950be98bf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:26:57.463729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d86ce757-6181-438d-a1da-4950be98bf12 None None] MFA Rules not processed for user `c3acafe7ed414c87a5ca235b96047aca`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:26:57.464919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d86ce757-6181-438d-a1da-4950be98bf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:26:57.471589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1140/2273] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:26:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:01.170691 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae49d7a-5ed4-4963-826a-8fe3b2cee407 service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:01.173228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fae49d7a-5ed4-4963-826a-8fe3b2cee407 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:01.244838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae49d7a-5ed4-4963-826a-8fe3b2cee407 service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:01.249732 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:01.251027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:01.251206 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:01.251318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] PATH_INFO: `/v3/users/b7b1959adde54f99bd2234b3a12bdd0a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:01.252564 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7b1959adde54f99bd2234b3a12bdd0a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:01.254678 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:01.312080 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7b1959adde54f99bd2234b3a12bdd0a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:01.313203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7b1959adde54f99bd2234b3a12bdd0a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:01.321366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7b1959adde54f99bd2234b3a12bdd0a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:01.329296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7b1959adde54f99bd2234b3a12bdd0a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:01.346368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7b1959adde54f99bd2234b3a12bdd0a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:01.353047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1134/2274] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:27:01 2022] DELETE /identity/v3/users/b7b1959adde54f99bd2234b3a12bdd0a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:01.358638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d32d-75ba-4e53-b417-350d3d4e21fd None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:01.359475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d32d-75ba-4e53-b417-350d3d4e21fd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:01.359754 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d32d-75ba-4e53-b417-350d3d4e21fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:01.380279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-cd86d32d-75ba-4e53-b417-350d3d4e21fd None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:01.381575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cd86d32d-75ba-4e53-b417-350d3d4e21fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:01.450643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1141/2275] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:01.461139 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:01.462629 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:01.536797 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a917d0f-b2f1-409a-add3-ee2296975913 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:01.546816 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102db9a5-57a0-4bec-9be9-9481e965c68a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:01.547565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102db9a5-57a0-4bec-9be9-9481e965c68a service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:01.547780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102db9a5-57a0-4bec-9be9-9481e965c68a service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:01.548143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102db9a5-57a0-4bec-9be9-9481e965c68a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:01.570327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102db9a5-57a0-4bec-9be9-9481e965c68a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:01.573822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102db9a5-57a0-4bec-9be9-9481e965c68a service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:01.587072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1135/2276] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:27:01.597827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0cecb-26bc-485a-9c1e-f7d770e8b936 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:01.598104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0cecb-26bc-485a-9c1e-f7d770e8b936 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:01.598774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0cecb-26bc-485a-9c1e-f7d770e8b936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:01.620978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d6a0cecb-26bc-485a-9c1e-f7d770e8b936 None None] MFA Rules not processed for user `d23a498bde00483c8fdd68dcde975e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:01.622297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d6a0cecb-26bc-485a-9c1e-f7d770e8b936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:01.686276 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1142/2277] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:02.205310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102db9a5-57a0-4bec-9be9-9481e965c68a service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:02.206932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-102db9a5-57a0-4bec-9be9-9481e965c68a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:02.224098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0cecb-26bc-485a-9c1e-f7d770e8b936 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:02.225544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d6a0cecb-26bc-485a-9c1e-f7d770e8b936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:02.252481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102db9a5-57a0-4bec-9be9-9481e965c68a service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:02.252659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0cecb-26bc-485a-9c1e-f7d770e8b936 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:02.263834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a95dcb-e714-426c-8736-ebdba726db55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:02.265895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a95dcb-e714-426c-8736-ebdba726db55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:02.267041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a95dcb-e714-426c-8736-ebdba726db55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:02.267170 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:02.267728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a95dcb-e714-426c-8736-ebdba726db55 admin admin] PATH_INFO: `/v3/users/a4efbc38f37340f3bec57265d7daf9d2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:02.268304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:02.268830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:02.269189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] PATH_INFO: `/v3/projects/5560ba31a85e4e6a9d04dbc5bd592c17` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:02.270097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a95dcb-e714-426c-8736-ebdba726db55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4efbc38f37340f3bec57265d7daf9d2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:02.270815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5560ba31a85e4e6a9d04dbc5bd592c17)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:02.274025 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:02.274870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a95dcb-e714-426c-8736-ebdba726db55 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:02.311829 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:02.353668 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e1a95dcb-e714-426c-8736-ebdba726db55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4efbc38f37340f3bec57265d7daf9d2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.363126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e1a95dcb-e714-426c-8736-ebdba726db55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4efbc38f37340f3bec57265d7daf9d2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.364532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e1a95dcb-e714-426c-8736-ebdba726db55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4efbc38f37340f3bec57265d7daf9d2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.379833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e1a95dcb-e714-426c-8736-ebdba726db55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4efbc38f37340f3bec57265d7daf9d2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.381590 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5560ba31a85e4e6a9d04dbc5bd592c17', 'request_id': 'req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.391707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5560ba31a85e4e6a9d04dbc5bd592c17', 'request_id': 'req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.398313 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e1a95dcb-e714-426c-8736-ebdba726db55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4efbc38f37340f3bec57265d7daf9d2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.401503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5560ba31a85e4e6a9d04dbc5bd592c17', 'request_id': 'req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.407866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1143/2278] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:27:02 2022] DELETE /identity/v3/users/a4efbc38f37340f3bec57265d7daf9d2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:02.412104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b86bc-4e82-4a9b-9fa7-7fb99f3d35d9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:02.412384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b86bc-4e82-4a9b-9fa7-7fb99f3d35d9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:02.412661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b86bc-4e82-4a9b-9fa7-7fb99f3d35d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:02.428794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] The token cache is being invalidate because project 5560ba31a85e4e6a9d04dbc5bd592c17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:27:02.429379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.430251 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.446111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1136/2279] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:27:02 2022] DELETE /identity/v3/projects/5560ba31a85e4e6a9d04dbc5bd592c17 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:02.446326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-385b86bc-4e82-4a9b-9fa7-7fb99f3d35d9 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:02.446326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-385b86bc-4e82-4a9b-9fa7-7fb99f3d35d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:02.457871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:02.459478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:02.463025 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:02.580828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1144/2280] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:27:02.593079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b86bc-4e82-4a9b-9fa7-7fb99f3d35d9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:02.595608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-385b86bc-4e82-4a9b-9fa7-7fb99f3d35d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:02.602521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe15e4f-9343-4a26-a511-9dbbeaee46fb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:02.612663 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab916523-e8a0-45ad-85f2-2d458f26ce50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:02.613515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab916523-e8a0-45ad-85f2-2d458f26ce50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:02.613893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab916523-e8a0-45ad-85f2-2d458f26ce50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:02.614313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab916523-e8a0-45ad-85f2-2d458f26ce50 admin admin] PATH_INFO: `/v3/users/18ee56757cf3470d88afea4ca0540bd1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:02.616392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab916523-e8a0-45ad-85f2-2d458f26ce50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18ee56757cf3470d88afea4ca0540bd1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:02.618517 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab916523-e8a0-45ad-85f2-2d458f26ce50 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:02.687116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ab916523-e8a0-45ad-85f2-2d458f26ce50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18ee56757cf3470d88afea4ca0540bd1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.687334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b86bc-4e82-4a9b-9fa7-7fb99f3d35d9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:02.688033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ab916523-e8a0-45ad-85f2-2d458f26ce50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18ee56757cf3470d88afea4ca0540bd1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.693778 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e7a70e-76b4-4a04-983f-77a20706b125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:02.694013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7a70e-76b4-4a04-983f-77a20706b125 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:02.694401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7a70e-76b4-4a04-983f-77a20706b125 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:02.695335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e7a70e-76b4-4a04-983f-77a20706b125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:02.697692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-72e7a70e-76b4-4a04-983f-77a20706b125 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:02.710207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ab916523-e8a0-45ad-85f2-2d458f26ce50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18ee56757cf3470d88afea4ca0540bd1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.716080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7a70e-76b4-4a04-983f-77a20706b125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:02.718246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e7a70e-76b4-4a04-983f-77a20706b125 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:02.727670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ab916523-e8a0-45ad-85f2-2d458f26ce50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18ee56757cf3470d88afea4ca0540bd1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.737616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1145/2281] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:27:02.753045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ab916523-e8a0-45ad-85f2-2d458f26ce50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18ee56757cf3470d88afea4ca0540bd1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:02.765887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1137/2282] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:27:02 2022] DELETE /identity/v3/users/18ee56757cf3470d88afea4ca0540bd1 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:27:02.770752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39fbeb-352f-4f39-b2ee-1bfe6a5e938f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:02.771053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39fbeb-352f-4f39-b2ee-1bfe6a5e938f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:02.771330 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39fbeb-352f-4f39-b2ee-1bfe6a5e938f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:02.829290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8a39fbeb-352f-4f39-b2ee-1bfe6a5e938f None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:02.830530 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8a39fbeb-352f-4f39-b2ee-1bfe6a5e938f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:02.966097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1146/2283] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:03.043940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82a7e1-2360-4c4e-bfc4-765d9309d02b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:03.044405 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82a7e1-2360-4c4e-bfc4-765d9309d02b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:03.044897 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82a7e1-2360-4c4e-bfc4-765d9309d02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:03.055058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39fbeb-352f-4f39-b2ee-1bfe6a5e938f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:03.056537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8a39fbeb-352f-4f39-b2ee-1bfe6a5e938f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:03.111366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5c82a7e1-2360-4c4e-bfc4-765d9309d02b None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:03.114589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5c82a7e1-2360-4c4e-bfc4-765d9309d02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:03.158612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:03.202142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39fbeb-352f-4f39-b2ee-1bfe6a5e938f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:03.206472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79505f-6a97-4f88-8bdc-55194f4f99b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:03.207101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79505f-6a97-4f88-8bdc-55194f4f99b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:03.207382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79505f-6a97-4f88-8bdc-55194f4f99b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:03.207649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79505f-6a97-4f88-8bdc-55194f4f99b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:03.209462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1138/2284] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:27:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:03.210413 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82a7e1-2360-4c4e-bfc4-765d9309d02b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:03.212666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5c82a7e1-2360-4c4e-bfc4-765d9309d02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:03.221744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79505f-6a97-4f88-8bdc-55194f4f99b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:03.224013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79505f-6a97-4f88-8bdc-55194f4f99b1 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:03.231469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:03.233370 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82a7e1-2360-4c4e-bfc4-765d9309d02b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:03.238604 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:03.239244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:03.239526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:03.239799 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce admin admin] PATH_INFO: `/v3/projects/61dd2124e7264f8f9cb4d52c5ef2dec1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:03.240979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=61dd2124e7264f8f9cb4d52c5ef2dec1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:03.242284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1147/2285] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:27:03.243010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:03.243597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79505f-6a97-4f88-8bdc-55194f4f99b1 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:03.245556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-db79505f-6a97-4f88-8bdc-55194f4f99b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:03.354304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61dd2124e7264f8f9cb4d52c5ef2dec1', 'request_id': 'req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:03.359376 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61dd2124e7264f8f9cb4d52c5ef2dec1', 'request_id': 'req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:03.370491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61dd2124e7264f8f9cb4d52c5ef2dec1', 'request_id': 'req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:03.373616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79505f-6a97-4f88-8bdc-55194f4f99b1 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:03.411015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce admin admin] The token cache is being invalidate because project 61dd2124e7264f8f9cb4d52c5ef2dec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:27:03.411015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:03.411616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e8ca0a3d-8760-4c33-92b7-e621740eb0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:03.413980 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1139/2286] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:27:03 2022] DELETE /identity/v3/projects/61dd2124e7264f8f9cb4d52c5ef2dec1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:03.418297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc68d9f-8849-4bb9-a09e-236e04ee6206 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:03.418723 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc68d9f-8849-4bb9-a09e-236e04ee6206 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:03.428420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc68d9f-8849-4bb9-a09e-236e04ee6206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:03.474521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1cc68d9f-8849-4bb9-a09e-236e04ee6206 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:03.476519 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1cc68d9f-8849-4bb9-a09e-236e04ee6206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:03.485577 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634b2ff-0fbf-4e37-a270-81c97e557d52 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:03.486341 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634b2ff-0fbf-4e37-a270-81c97e557d52 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:03.486543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634b2ff-0fbf-4e37-a270-81c97e557d52 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:03.486814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634b2ff-0fbf-4e37-a270-81c97e557d52 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:03.495595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9634b2ff-0fbf-4e37-a270-81c97e557d52 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:03.604251 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1140/2287] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:03.613078 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12d003-1d66-4627-96f8-dca3823f20ac None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:03.613539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12d003-1d66-4627-96f8-dca3823f20ac None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:03.614011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12d003-1d66-4627-96f8-dca3823f20ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:03.648621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0b12d003-1d66-4627-96f8-dca3823f20ac None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:03.651154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0b12d003-1d66-4627-96f8-dca3823f20ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:03.666842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1141/2288] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:03.666842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12d003-1d66-4627-96f8-dca3823f20ac None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:03.668047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0b12d003-1d66-4627-96f8-dca3823f20ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:03.672818 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634b2ff-0fbf-4e37-a270-81c97e557d52 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:03.676563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634b2ff-0fbf-4e37-a270-81c97e557d52 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:03.688296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1148/2289] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:27:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:27:03.689186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634b2ff-0fbf-4e37-a270-81c97e557d52 service swift] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:03.694253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12d003-1d66-4627-96f8-dca3823f20ac None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:03.702172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634b2ff-0fbf-4e37-a270-81c97e557d52 service swift] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:03.707702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:03.708602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:03.708602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:03.708752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20199eb-203d-4559-9f89-83b8c9c80e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:03.708851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] PATH_INFO: `/v3/projects/879e23eef9ae49a1950fb25401173647` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:03.709429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20199eb-203d-4559-9f89-83b8c9c80e1b admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:03.709649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20199eb-203d-4559-9f89-83b8c9c80e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:03.709929 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20199eb-203d-4559-9f89-83b8c9c80e1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:03.710648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20199eb-203d-4559-9f89-83b8c9c80e1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:27:03.710920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20199eb-203d-4559-9f89-83b8c9c80e1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:03.711199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=879e23eef9ae49a1950fb25401173647)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:03.712530 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20199eb-203d-4559-9f89-83b8c9c80e1b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:03.715072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:03.726809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1149/2290] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:27:03.735782 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20199eb-203d-4559-9f89-83b8c9c80e1b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:03.764446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20199eb-203d-4559-9f89-83b8c9c80e1b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:03.775917 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b87d9-69f7-4cef-8948-5213ad836600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:03.776913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b87d9-69f7-4cef-8948-5213ad836600 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:03.776913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b87d9-69f7-4cef-8948-5213ad836600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:03.777161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b87d9-69f7-4cef-8948-5213ad836600 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:03.777692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b87d9-69f7-4cef-8948-5213ad836600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:03.779762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b87d9-69f7-4cef-8948-5213ad836600 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:03.819870 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:03.834599 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '879e23eef9ae49a1950fb25401173647', 'request_id': 'req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:03.843434 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '879e23eef9ae49a1950fb25401173647', 'request_id': 'req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:03.854811 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1150/2291] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:27:03 2022] POST /identity/v3/projects => generated 363 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:27:03.855032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '879e23eef9ae49a1950fb25401173647', 'request_id': 'req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:03.856776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b87d9-69f7-4cef-8948-5213ad836600 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:03.883748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:03.886102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] The token cache is being invalidate because project 879e23eef9ae49a1950fb25401173647 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:27:03.888286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:03.891876 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:03.898461 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1142/2292] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:27:03 2022] DELETE /identity/v3/projects/879e23eef9ae49a1950fb25401173647 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:03.905454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:03.912521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:04.025501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b87d9-69f7-4cef-8948-5213ad836600 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:04.032004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8a8e9-b71f-4abf-9a9d-c09cb088b8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:04.032659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8a8e9-b71f-4abf-9a9d-c09cb088b8c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:04.033068 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8a8e9-b71f-4abf-9a9d-c09cb088b8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:04.033356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8a8e9-b71f-4abf-9a9d-c09cb088b8c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:04.034113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8a8e9-b71f-4abf-9a9d-c09cb088b8c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:04.036795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c8a8e9-b71f-4abf-9a9d-c09cb088b8c2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:04.079190 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1151/2293] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:27:03 2022] POST /identity/v3/users => generated 282 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:27:04.085513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8a8e9-b71f-4abf-9a9d-c09cb088b8c2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:04.090088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-00c8a8e9-b71f-4abf-9a9d-c09cb088b8c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:04.092658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42340b-5d99-48b9-a6b6-9f526c0cfa0b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:04.105165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc1a54-703e-4023-93e3-137e1382f19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:04.108027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc1a54-703e-4023-93e3-137e1382f19d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:04.108433 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc1a54-703e-4023-93e3-137e1382f19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:04.108894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc1a54-703e-4023-93e3-137e1382f19d admin admin] PATH_INFO: `/v3/users/5149f40aabb34f9aabdd632013fe6ed8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:04.113248 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc1a54-703e-4023-93e3-137e1382f19d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5149f40aabb34f9aabdd632013fe6ed8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:04.115600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc1a54-703e-4023-93e3-137e1382f19d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:04.137619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8a8e9-b71f-4abf-9a9d-c09cb088b8c2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:04.157565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4931a-5078-4eb4-a15e-a9a6121754f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:04.159382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4931a-5078-4eb4-a15e-a9a6121754f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:04.160662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4931a-5078-4eb4-a15e-a9a6121754f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:04.161835 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4931a-5078-4eb4-a15e-a9a6121754f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:04.162787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4931a-5078-4eb4-a15e-a9a6121754f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:04.168402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4931a-5078-4eb4-a15e-a9a6121754f0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:04.203167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1152/2294] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:27:04.207576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4931a-5078-4eb4-a15e-a9a6121754f0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:04.224578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4931a-5078-4eb4-a15e-a9a6121754f0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:04.232710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b3bc1a54-703e-4023-93e3-137e1382f19d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5149f40aabb34f9aabdd632013fe6ed8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:04.235466 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02614226-66c5-43d5-96ed-03da1a9a1fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:04.236186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02614226-66c5-43d5-96ed-03da1a9a1fae admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:04.236472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02614226-66c5-43d5-96ed-03da1a9a1fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:04.236775 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b3bc1a54-703e-4023-93e3-137e1382f19d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5149f40aabb34f9aabdd632013fe6ed8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:04.238798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02614226-66c5-43d5-96ed-03da1a9a1fae admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:04.239305 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02614226-66c5-43d5-96ed-03da1a9a1fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:04.241166 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02614226-66c5-43d5-96ed-03da1a9a1fae admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:04.251009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b3bc1a54-703e-4023-93e3-137e1382f19d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5149f40aabb34f9aabdd632013fe6ed8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:04.263103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b3bc1a54-703e-4023-93e3-137e1382f19d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5149f40aabb34f9aabdd632013fe6ed8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:04.269607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1153/2295] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:27:04.275032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02614226-66c5-43d5-96ed-03da1a9a1fae admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:04.293745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-02614226-66c5-43d5-96ed-03da1a9a1fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:04.319812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b3bc1a54-703e-4023-93e3-137e1382f19d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5149f40aabb34f9aabdd632013fe6ed8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:04.327948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02614226-66c5-43d5-96ed-03da1a9a1fae admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:04.332296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1143/2296] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:27:03 2022] DELETE /identity/v3/users/5149f40aabb34f9aabdd632013fe6ed8 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:27:04.346490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdf75d-7705-4923-9365-3a921d2e8e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:04.347097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdf75d-7705-4923-9365-3a921d2e8e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:04.348304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdf75d-7705-4923-9365-3a921d2e8e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:04.348304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bdf75d-7705-4923-9365-3a921d2e8e0a admin admin] PATH_INFO: `/v3/projects/438e2c1f93bd41a7bd517a950b06d6ff/users/7d1bc9100d5a4d1fa28bbcac495fbfa7/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:04.425132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bdf75d-7705-4923-9365-3a921d2e8e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=438e2c1f93bd41a7bd517a950b06d6ff, user_id=7d1bc9100d5a4d1fa28bbcac495fbfa7, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:04.429924 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bdf75d-7705-4923-9365-3a921d2e8e0a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:04.460923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1154/2297] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:27:04 2022] PUT /identity/v3/projects/438e2c1f93bd41a7bd517a950b06d6ff/users/7d1bc9100d5a4d1fa28bbcac495fbfa7/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:04.482099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e93629-f605-4895-bb43-8f676e76fef3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:04.482099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e93629-f605-4895-bb43-8f676e76fef3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:04.482099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e93629-f605-4895-bb43-8f676e76fef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:04.561232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-92e93629-f605-4895-bb43-8f676e76fef3 None None] MFA Rules not processed for user `7d1bc9100d5a4d1fa28bbcac495fbfa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:04.573891 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-92e93629-f605-4895-bb43-8f676e76fef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:04.707260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:04.811296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1144/2298] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:04.819414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdf75d-7705-4923-9365-3a921d2e8e0a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:04.821105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-52bdf75d-7705-4923-9365-3a921d2e8e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:04.946244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bdf75d-7705-4923-9365-3a921d2e8e0a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:04.955455 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d76cca-de5d-4677-83bf-7a602ee194b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:04.956791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d76cca-de5d-4677-83bf-7a602ee194b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:04.957073 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d76cca-de5d-4677-83bf-7a602ee194b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:04.957348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d76cca-de5d-4677-83bf-7a602ee194b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:04.983949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d76cca-de5d-4677-83bf-7a602ee194b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:04.986015 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d76cca-de5d-4677-83bf-7a602ee194b4 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:05.020024 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1155/2299] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:27:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:27:05.342423 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e93629-f605-4895-bb43-8f676e76fef3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:05.348438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-92e93629-f605-4895-bb43-8f676e76fef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:05.517060 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e93629-f605-4895-bb43-8f676e76fef3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:05.534653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1a13e-261e-4a6e-a075-30ef56c31719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:05.534653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1a13e-261e-4a6e-a075-30ef56c31719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:05.534653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1a13e-261e-4a6e-a075-30ef56c31719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:05.536106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1a13e-261e-4a6e-a075-30ef56c31719 admin admin] PATH_INFO: `/v3/projects/141d8e7566f444a6857cf96560b8b68b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:05.544594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1a13e-261e-4a6e-a075-30ef56c31719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=141d8e7566f444a6857cf96560b8b68b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:05.544594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1a13e-261e-4a6e-a075-30ef56c31719 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:05.675596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ec1a13e-261e-4a6e-a075-30ef56c31719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141d8e7566f444a6857cf96560b8b68b', 'request_id': 'req-4ec1a13e-261e-4a6e-a075-30ef56c31719'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:05.688670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ec1a13e-261e-4a6e-a075-30ef56c31719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141d8e7566f444a6857cf96560b8b68b', 'request_id': 'req-4ec1a13e-261e-4a6e-a075-30ef56c31719'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:05.708791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ec1a13e-261e-4a6e-a075-30ef56c31719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141d8e7566f444a6857cf96560b8b68b', 'request_id': 'req-4ec1a13e-261e-4a6e-a075-30ef56c31719'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:05.748662 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ec1a13e-261e-4a6e-a075-30ef56c31719 admin admin] The token cache is being invalidate because project 141d8e7566f444a6857cf96560b8b68b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:27:05.749043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ec1a13e-261e-4a6e-a075-30ef56c31719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:05.751550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ec1a13e-261e-4a6e-a075-30ef56c31719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:05.758213 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1145/2300] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:27:05 2022] DELETE /identity/v3/projects/141d8e7566f444a6857cf96560b8b68b => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:05.760070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319783dd-29c4-4094-a726-e658c3c95b22 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:05.760560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319783dd-29c4-4094-a726-e658c3c95b22 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:05.760658 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319783dd-29c4-4094-a726-e658c3c95b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:05.788100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-319783dd-29c4-4094-a726-e658c3c95b22 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:05.791528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-319783dd-29c4-4094-a726-e658c3c95b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:05.881924 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:05.917976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1156/2301] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:27:05.926304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79b95e-f5df-44c3-99c2-9f00d095f713 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:05.926589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79b95e-f5df-44c3-99c2-9f00d095f713 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:05.926864 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79b95e-f5df-44c3-99c2-9f00d095f713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:05.965015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3e79b95e-f5df-44c3-99c2-9f00d095f713 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:05.966297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3e79b95e-f5df-44c3-99c2-9f00d095f713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:05.992969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1146/2302] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:05.994833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319783dd-29c4-4094-a726-e658c3c95b22 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:06.010564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319783dd-29c4-4094-a726-e658c3c95b22 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:06.024386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f701b-4f76-4edf-b6bc-cd3b43a1c9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:06.027011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f701b-4f76-4edf-b6bc-cd3b43a1c9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:06.027872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f701b-4f76-4edf-b6bc-cd3b43a1c9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:06.028586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f701b-4f76-4edf-b6bc-cd3b43a1c9a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:06.029665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f701b-4f76-4edf-b6bc-cd3b43a1c9a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:27:06.030131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f701b-4f76-4edf-b6bc-cd3b43a1c9a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:06.032202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f701b-4f76-4edf-b6bc-cd3b43a1c9a7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:06.055165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1157/2303] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:27:06.060117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79b95e-f5df-44c3-99c2-9f00d095f713 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:06.070429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79b95e-f5df-44c3-99c2-9f00d095f713 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:06.089319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20444e24-bea3-439a-9084-b890d3d35f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:06.089778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20444e24-bea3-439a-9084-b890d3d35f15 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:06.090058 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20444e24-bea3-439a-9084-b890d3d35f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:06.090351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20444e24-bea3-439a-9084-b890d3d35f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:06.091114 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20444e24-bea3-439a-9084-b890d3d35f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:06.093041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20444e24-bea3-439a-9084-b890d3d35f15 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:06.148803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1147/2304] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:27:06 2022] POST /identity/v3/projects => generated 387 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:27:06.154009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f701b-4f76-4edf-b6bc-cd3b43a1c9a7 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:06.168165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f701b-4f76-4edf-b6bc-cd3b43a1c9a7 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:06.272716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ade165-a815-4e62-87d2-34c7435cc870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:06.273335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ade165-a815-4e62-87d2-34c7435cc870 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:06.273613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ade165-a815-4e62-87d2-34c7435cc870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:06.273881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ade165-a815-4e62-87d2-34c7435cc870 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:06.274632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ade165-a815-4e62-87d2-34c7435cc870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:06.279070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ade165-a815-4e62-87d2-34c7435cc870 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:06.322709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1158/2305] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:27:06 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:27:06.326474 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20444e24-bea3-439a-9084-b890d3d35f15 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:06.339523 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20444e24-bea3-439a-9084-b890d3d35f15 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:06.345214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ad221-df49-44f0-b698-d0a9131bbb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:06.345903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ad221-df49-44f0-b698-d0a9131bbb92 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:06.346219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ad221-df49-44f0-b698-d0a9131bbb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:06.346550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ad221-df49-44f0-b698-d0a9131bbb92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:06.347270 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ad221-df49-44f0-b698-d0a9131bbb92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:06.350417 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ad221-df49-44f0-b698-d0a9131bbb92 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:06.362446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1148/2306] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:27:06.366260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ade165-a815-4e62-87d2-34c7435cc870 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:06.378237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ade165-a815-4e62-87d2-34c7435cc870 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:06.391536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1aa31-e8bb-4dd3-b31f-5f248f7aaafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:06.392303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1aa31-e8bb-4dd3-b31f-5f248f7aaafc admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:06.392677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1aa31-e8bb-4dd3-b31f-5f248f7aaafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:06.393042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1aa31-e8bb-4dd3-b31f-5f248f7aaafc admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:06.393795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1aa31-e8bb-4dd3-b31f-5f248f7aaafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:06.395699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1aa31-e8bb-4dd3-b31f-5f248f7aaafc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:06.414200 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ad221-df49-44f0-b698-d0a9131bbb92 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:06.414511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1159/2307] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:27:06.428538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ad221-df49-44f0-b698-d0a9131bbb92 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:06.439427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669ee07-4ea3-455d-8f8f-2a03ce19374e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:06.440108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669ee07-4ea3-455d-8f8f-2a03ce19374e admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:06.440385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669ee07-4ea3-455d-8f8f-2a03ce19374e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:06.441087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669ee07-4ea3-455d-8f8f-2a03ce19374e admin admin] PATH_INFO: `/v3/projects/45fac1cd680a4336b98109ac2b97df6d/users/f38dfed02c7c45ff9ec5ff972f7e565d/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:06.479583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a669ee07-4ea3-455d-8f8f-2a03ce19374e admin admin] RBAC: Authorizing `identity:create_grant(project_id=45fac1cd680a4336b98109ac2b97df6d, user_id=f38dfed02c7c45ff9ec5ff972f7e565d, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:06.483538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a669ee07-4ea3-455d-8f8f-2a03ce19374e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:06.504329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1149/2308] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:27:06 2022] PUT /identity/v3/projects/45fac1cd680a4336b98109ac2b97df6d/users/f38dfed02c7c45ff9ec5ff972f7e565d/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:27:06.517550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46c33a-275b-4283-9492-8e2380646a5f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:06.517864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46c33a-275b-4283-9492-8e2380646a5f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:06.518114 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46c33a-275b-4283-9492-8e2380646a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:06.576839 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ee46c33a-275b-4283-9492-8e2380646a5f None None] MFA Rules not processed for user `f38dfed02c7c45ff9ec5ff972f7e565d`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:06.578153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ee46c33a-275b-4283-9492-8e2380646a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:06.694489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1160/2309] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:06.705723 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669ee07-4ea3-455d-8f8f-2a03ce19374e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:06.713452 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:06.780153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669ee07-4ea3-455d-8f8f-2a03ce19374e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:06.785690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d9ff3-0e41-4a51-bc5d-235f8bc2e309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:06.786587 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d9ff3-0e41-4a51-bc5d-235f8bc2e309 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:06.786866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d9ff3-0e41-4a51-bc5d-235f8bc2e309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:06.787275 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d9ff3-0e41-4a51-bc5d-235f8bc2e309 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:06.787977 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d9ff3-0e41-4a51-bc5d-235f8bc2e309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:06.790355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d9ff3-0e41-4a51-bc5d-235f8bc2e309 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:06.818659 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1150/2310] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:27:06 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:27:06.822880 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46c33a-275b-4283-9492-8e2380646a5f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:06.833788 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46c33a-275b-4283-9492-8e2380646a5f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:06.891018 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:06.904699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc424e-6c41-4be8-8631-ad1f6f09c037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:06.905300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc424e-6c41-4be8-8631-ad1f6f09c037 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:06.905576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc424e-6c41-4be8-8631-ad1f6f09c037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:06.905842 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc424e-6c41-4be8-8631-ad1f6f09c037 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:06.906514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc424e-6c41-4be8-8631-ad1f6f09c037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:06.908416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc424e-6c41-4be8-8631-ad1f6f09c037 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:06.938635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1161/2311] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:27:06 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:27:06.940044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d9ff3-0e41-4a51-bc5d-235f8bc2e309 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:06.954347 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d9ff3-0e41-4a51-bc5d-235f8bc2e309 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:06.959594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d44405-3cbf-433f-a3f7-cdfc59eaa55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:06.960321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d44405-3cbf-433f-a3f7-cdfc59eaa55d admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:06.960701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d44405-3cbf-433f-a3f7-cdfc59eaa55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:06.961069 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d44405-3cbf-433f-a3f7-cdfc59eaa55d admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:06.961942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d44405-3cbf-433f-a3f7-cdfc59eaa55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:06.964299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d44405-3cbf-433f-a3f7-cdfc59eaa55d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:06.976979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1151/2312] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:27:06.981586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc424e-6c41-4be8-8631-ad1f6f09c037 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:06.992761 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc424e-6c41-4be8-8631-ad1f6f09c037 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:06.997879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00138a9-1c2e-407f-b1bb-2ca2e0028a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:06.998577 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00138a9-1c2e-407f-b1bb-2ca2e0028a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:06.998876 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00138a9-1c2e-407f-b1bb-2ca2e0028a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:06.999150 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00138a9-1c2e-407f-b1bb-2ca2e0028a8a admin admin] PATH_INFO: `/v3/projects/a5cd76a45fd9471eb438fef70c62b572/users/2b7540c1c0c64401833195579efc4b1b/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:07.019957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00138a9-1c2e-407f-b1bb-2ca2e0028a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5cd76a45fd9471eb438fef70c62b572, user_id=2b7540c1c0c64401833195579efc4b1b, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:07.023396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00138a9-1c2e-407f-b1bb-2ca2e0028a8a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:07.039618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1162/2313] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:27:06 2022] PUT /identity/v3/projects/a5cd76a45fd9471eb438fef70c62b572/users/2b7540c1c0c64401833195579efc4b1b/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:27:07.043721 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d44405-3cbf-433f-a3f7-cdfc59eaa55d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:07.057313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d44405-3cbf-433f-a3f7-cdfc59eaa55d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:07.135381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2abb71-3eb6-4395-adfa-aa51c15379d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:07.136116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2abb71-3eb6-4395-adfa-aa51c15379d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:07.136392 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2abb71-3eb6-4395-adfa-aa51c15379d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:07.136659 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2abb71-3eb6-4395-adfa-aa51c15379d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:07.137301 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2abb71-3eb6-4395-adfa-aa51c15379d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:07.138940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2abb71-3eb6-4395-adfa-aa51c15379d4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:07.151336 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1152/2314] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:27:07.155606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00138a9-1c2e-407f-b1bb-2ca2e0028a8a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:07.166634 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00138a9-1c2e-407f-b1bb-2ca2e0028a8a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:07.171454 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d854a-79f5-4dfb-ba70-d72bfbe2527d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:07.172108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d854a-79f5-4dfb-ba70-d72bfbe2527d admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:07.172402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d854a-79f5-4dfb-ba70-d72bfbe2527d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:07.172676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d854a-79f5-4dfb-ba70-d72bfbe2527d admin admin] PATH_INFO: `/v3/projects/a5cd76a45fd9471eb438fef70c62b572/users/2b7540c1c0c64401833195579efc4b1b/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:07.175037 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d854a-79f5-4dfb-ba70-d72bfbe2527d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5cd76a45fd9471eb438fef70c62b572, user_id=2b7540c1c0c64401833195579efc4b1b, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:07.177684 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d854a-79f5-4dfb-ba70-d72bfbe2527d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:07.189242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1163/2315] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:27:07 2022] PUT /identity/v3/projects/a5cd76a45fd9471eb438fef70c62b572/users/2b7540c1c0c64401833195579efc4b1b/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:07.199870 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d37fa-6672-46cb-9828-ad6dcdc41b2c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:07.200145 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d37fa-6672-46cb-9828-ad6dcdc41b2c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:07.200414 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d37fa-6672-46cb-9828-ad6dcdc41b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:07.230407 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-828d37fa-6672-46cb-9828-ad6dcdc41b2c None None] MFA Rules not processed for user `2b7540c1c0c64401833195579efc4b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:07.231882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-828d37fa-6672-46cb-9828-ad6dcdc41b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:07.380106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:07.390471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1153/2316] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:07 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:07.398846 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d854a-79f5-4dfb-ba70-d72bfbe2527d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:07.400648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-193d854a-79f5-4dfb-ba70-d72bfbe2527d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:07.481273 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d854a-79f5-4dfb-ba70-d72bfbe2527d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:07.484950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f42752-74c0-4209-85bd-5582d46ee59d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:07.485552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f42752-74c0-4209-85bd-5582d46ee59d service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:07.485830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f42752-74c0-4209-85bd-5582d46ee59d service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:07.486154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f42752-74c0-4209-85bd-5582d46ee59d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:07.496926 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f42752-74c0-4209-85bd-5582d46ee59d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:07.498511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f42752-74c0-4209-85bd-5582d46ee59d service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:07.508702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1164/2317] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:27:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:27:07.594144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d37fa-6672-46cb-9828-ad6dcdc41b2c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:07.673360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d37fa-6672-46cb-9828-ad6dcdc41b2c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:07.677846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:07.678575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:07.678772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:07.679037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] PATH_INFO: `/v3/users/f38dfed02c7c45ff9ec5ff972f7e565d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:07.680139 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38dfed02c7c45ff9ec5ff972f7e565d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:07.682091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:07.756891 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38dfed02c7c45ff9ec5ff972f7e565d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:07.757707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38dfed02c7c45ff9ec5ff972f7e565d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:07.776000 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38dfed02c7c45ff9ec5ff972f7e565d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:07.787773 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38dfed02c7c45ff9ec5ff972f7e565d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:07.815640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38dfed02c7c45ff9ec5ff972f7e565d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:07.823628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1154/2318] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:27:07 2022] DELETE /identity/v3/users/f38dfed02c7c45ff9ec5ff972f7e565d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:07.827962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76d093-f521-48e5-8e3c-ee4dd3c31cc7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:07.828442 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76d093-f521-48e5-8e3c-ee4dd3c31cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:07.828548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76d093-f521-48e5-8e3c-ee4dd3c31cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:07.850286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5b76d093-f521-48e5-8e3c-ee4dd3c31cc7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:07.851505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5b76d093-f521-48e5-8e3c-ee4dd3c31cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:07.925118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1165/2319] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:07.933099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:07.934707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:08.017668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef086907-093b-4476-acfd-8d12126603f2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:08.021324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44510c-fc70-48d3-9f2d-70939182400c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:08.022274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44510c-fc70-48d3-9f2d-70939182400c service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:08.029943 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44510c-fc70-48d3-9f2d-70939182400c service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:08.030254 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44510c-fc70-48d3-9f2d-70939182400c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:08.043741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44510c-fc70-48d3-9f2d-70939182400c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:08.045626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44510c-fc70-48d3-9f2d-70939182400c service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:08.060088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1155/2320] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:27:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:27:08.482793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76d093-f521-48e5-8e3c-ee4dd3c31cc7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:08.484712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5b76d093-f521-48e5-8e3c-ee4dd3c31cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:08.513543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76d093-f521-48e5-8e3c-ee4dd3c31cc7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:08.523932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:08.524592 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:08.524954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:08.528532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] PATH_INFO: `/v3/projects/45fac1cd680a4336b98109ac2b97df6d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:08.533713 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45fac1cd680a4336b98109ac2b97df6d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:08.538613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:08.611596 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:08.631356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45fac1cd680a4336b98109ac2b97df6d', 'request_id': 'req-b2a3546a-d3b5-468b-87b7-837a3adb70f6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:08.644984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45fac1cd680a4336b98109ac2b97df6d', 'request_id': 'req-b2a3546a-d3b5-468b-87b7-837a3adb70f6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:08.658141 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45fac1cd680a4336b98109ac2b97df6d', 'request_id': 'req-b2a3546a-d3b5-468b-87b7-837a3adb70f6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:08.675470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] The token cache is being invalidate because project 45fac1cd680a4336b98109ac2b97df6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:27:08.675769 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:08.676538 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:08.679822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1166/2321] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:27:08 2022] DELETE /identity/v3/projects/45fac1cd680a4336b98109ac2b97df6d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:08.690568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44510c-fc70-48d3-9f2d-70939182400c service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:08.704449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44510c-fc70-48d3-9f2d-70939182400c service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:08.725317 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:08.775761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2227db-1472-4dcb-b0a8-f5e15d4188f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:08.776393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2227db-1472-4dcb-b0a8-f5e15d4188f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:08.776667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2227db-1472-4dcb-b0a8-f5e15d4188f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:08.776938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2227db-1472-4dcb-b0a8-f5e15d4188f9 admin admin] PATH_INFO: `/v3/users/2b7540c1c0c64401833195579efc4b1b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:08.778601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2227db-1472-4dcb-b0a8-f5e15d4188f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b7540c1c0c64401833195579efc4b1b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:08.780621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2227db-1472-4dcb-b0a8-f5e15d4188f9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:08.862286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-af2227db-1472-4dcb-b0a8-f5e15d4188f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b7540c1c0c64401833195579efc4b1b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:08.863007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-af2227db-1472-4dcb-b0a8-f5e15d4188f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b7540c1c0c64401833195579efc4b1b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:08.887805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-af2227db-1472-4dcb-b0a8-f5e15d4188f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b7540c1c0c64401833195579efc4b1b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:08.906105 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-af2227db-1472-4dcb-b0a8-f5e15d4188f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b7540c1c0c64401833195579efc4b1b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:08.932406 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-af2227db-1472-4dcb-b0a8-f5e15d4188f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b7540c1c0c64401833195579efc4b1b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:08.939932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1156/2322] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:27:08 2022] DELETE /identity/v3/users/2b7540c1c0c64401833195579efc4b1b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:09.236030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:09.237583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:09.263105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3546a-d3b5-468b-87b7-837a3adb70f6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:09.337708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:09.339414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:09.339769 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:09.340134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1 admin admin] PATH_INFO: `/v3/projects/a5cd76a45fd9471eb438fef70c62b572` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:09.341583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5cd76a45fd9471eb438fef70c62b572)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:09.344538 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:09.407722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5cd76a45fd9471eb438fef70c62b572', 'request_id': 'req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:09.413289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5cd76a45fd9471eb438fef70c62b572', 'request_id': 'req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:09.420676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5cd76a45fd9471eb438fef70c62b572', 'request_id': 'req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:09.437925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1 admin admin] The token cache is being invalidate because project a5cd76a45fd9471eb438fef70c62b572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:27:09.438679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:09.439511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2c54e6b5-44e4-4c52-a5f0-c6cef9f907d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:09.443149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1167/2323] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:27:09 2022] DELETE /identity/v3/projects/a5cd76a45fd9471eb438fef70c62b572 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:09.448727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc352758-0ed4-45ad-9d66-daf8ef838bbe None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:09.449013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc352758-0ed4-45ad-9d66-daf8ef838bbe None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:09.449286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc352758-0ed4-45ad-9d66-daf8ef838bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:09.477772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-cc352758-0ed4-45ad-9d66-daf8ef838bbe None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:09.479317 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cc352758-0ed4-45ad-9d66-daf8ef838bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:09.566142 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1157/2324] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:27:09.574541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc1e5a-8fb0-49b4-bef5-20b2e2ded8bb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:09.574804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc1e5a-8fb0-49b4-bef5-20b2e2ded8bb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:09.575086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc1e5a-8fb0-49b4-bef5-20b2e2ded8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:09.640365 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-80bc1e5a-8fb0-49b4-bef5-20b2e2ded8bb None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:09.642855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-80bc1e5a-8fb0-49b4-bef5-20b2e2ded8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:09.654117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1168/2325] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:09.656273 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc352758-0ed4-45ad-9d66-daf8ef838bbe None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:09.667070 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc352758-0ed4-45ad-9d66-daf8ef838bbe None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:09.673180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0afc36-c411-48c9-9cb6-4d31a5d3f36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:09.673855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0afc36-c411-48c9-9cb6-4d31a5d3f36e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:09.674154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0afc36-c411-48c9-9cb6-4d31a5d3f36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:09.674421 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0afc36-c411-48c9-9cb6-4d31a5d3f36e admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:09.675073 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0afc36-c411-48c9-9cb6-4d31a5d3f36e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:27:09.675371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0afc36-c411-48c9-9cb6-4d31a5d3f36e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:09.676986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0afc36-c411-48c9-9cb6-4d31a5d3f36e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:09.698241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc1e5a-8fb0-49b4-bef5-20b2e2ded8bb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:09.705006 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1158/2326] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:27:09.709012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc1e5a-8fb0-49b4-bef5-20b2e2ded8bb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:09.713949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e1fdb-0803-40a6-b55e-19b49c07056a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:09.714708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e1fdb-0803-40a6-b55e-19b49c07056a admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:09.715161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e1fdb-0803-40a6-b55e-19b49c07056a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:09.715477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e1fdb-0803-40a6-b55e-19b49c07056a admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:09.718194 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e1fdb-0803-40a6-b55e-19b49c07056a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:09.721113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e1fdb-0803-40a6-b55e-19b49c07056a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:09.775190 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1169/2327] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:27:09 2022] POST /identity/v3/projects => generated 389 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:27:09.779470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0afc36-c411-48c9-9cb6-4d31a5d3f36e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:09.794892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0afc36-c411-48c9-9cb6-4d31a5d3f36e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:09.835817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:09.890646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89c350-4ba6-4742-a36e-2fe653289868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:09.891265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89c350-4ba6-4742-a36e-2fe653289868 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:09.891542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89c350-4ba6-4742-a36e-2fe653289868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:09.891817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89c350-4ba6-4742-a36e-2fe653289868 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:09.892460 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89c350-4ba6-4742-a36e-2fe653289868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:09.894406 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89c350-4ba6-4742-a36e-2fe653289868 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:09.922804 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1159/2328] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:27:09 2022] POST /identity/v3/users => generated 295 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:27:09.926730 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e1fdb-0803-40a6-b55e-19b49c07056a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:09.953360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e1fdb-0803-40a6-b55e-19b49c07056a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:09.966930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c51ee-de31-4ff2-8650-7527d14fe272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:09.967746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c51ee-de31-4ff2-8650-7527d14fe272 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:09.968036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c51ee-de31-4ff2-8650-7527d14fe272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:09.973356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c51ee-de31-4ff2-8650-7527d14fe272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:09.974098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c51ee-de31-4ff2-8650-7527d14fe272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:09.975753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c51ee-de31-4ff2-8650-7527d14fe272 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:09.999443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1170/2329] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:27:10.003177 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89c350-4ba6-4742-a36e-2fe653289868 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:10.013892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89c350-4ba6-4742-a36e-2fe653289868 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:10.030630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f947ed6-fd93-4657-90b4-e9a75e68a957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:10.031528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f947ed6-fd93-4657-90b4-e9a75e68a957 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:10.031958 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f947ed6-fd93-4657-90b4-e9a75e68a957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:10.032513 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f947ed6-fd93-4657-90b4-e9a75e68a957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:10.033293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f947ed6-fd93-4657-90b4-e9a75e68a957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:10.035081 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f947ed6-fd93-4657-90b4-e9a75e68a957 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:10.072182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1160/2330] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:27:10.077975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c51ee-de31-4ff2-8650-7527d14fe272 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:10.108079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c51ee-de31-4ff2-8650-7527d14fe272 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:10.114038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d459293-f30e-4909-b038-039a312ad3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:10.114825 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459293-f30e-4909-b038-039a312ad3b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:10.115163 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459293-f30e-4909-b038-039a312ad3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:10.115542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459293-f30e-4909-b038-039a312ad3b3 admin admin] PATH_INFO: `/v3/projects/ac375b425c5e4fffb3c380a738988795/users/02a238f1347c42719b02611996730b02/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:10.132533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d459293-f30e-4909-b038-039a312ad3b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac375b425c5e4fffb3c380a738988795, user_id=02a238f1347c42719b02611996730b02, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:10.135502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d459293-f30e-4909-b038-039a312ad3b3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:10.138239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:10.152315 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1171/2331] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:27:10 2022] PUT /identity/v3/projects/ac375b425c5e4fffb3c380a738988795/users/02a238f1347c42719b02611996730b02/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:27:10.163815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63643350-75ef-4d76-9778-e2cbc0bb2417 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:10.164186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63643350-75ef-4d76-9778-e2cbc0bb2417 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:10.164530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63643350-75ef-4d76-9778-e2cbc0bb2417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:10.199182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-63643350-75ef-4d76-9778-e2cbc0bb2417 None None] MFA Rules not processed for user `02a238f1347c42719b02611996730b02`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:10.200485 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-63643350-75ef-4d76-9778-e2cbc0bb2417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:10.284472 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1161/2332] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:10.289097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d459293-f30e-4909-b038-039a312ad3b3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:10.364633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d459293-f30e-4909-b038-039a312ad3b3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:10.369521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d75ae-4f43-49e4-8f1f-7123026ee37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:10.370222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d75ae-4f43-49e4-8f1f-7123026ee37e admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:10.370501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d75ae-4f43-49e4-8f1f-7123026ee37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:10.370775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d75ae-4f43-49e4-8f1f-7123026ee37e admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:10.371416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d75ae-4f43-49e4-8f1f-7123026ee37e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:10.373374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d75ae-4f43-49e4-8f1f-7123026ee37e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:10.407562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1172/2333] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:27:10 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:27:10.411671 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63643350-75ef-4d76-9778-e2cbc0bb2417 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:10.424038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63643350-75ef-4d76-9778-e2cbc0bb2417 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:10.484959 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c69569-b439-43ac-b786-cd858fe5d880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:10.485576 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c69569-b439-43ac-b786-cd858fe5d880 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:10.485851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c69569-b439-43ac-b786-cd858fe5d880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:10.486138 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c69569-b439-43ac-b786-cd858fe5d880 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:10.486771 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c69569-b439-43ac-b786-cd858fe5d880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:10.488704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c69569-b439-43ac-b786-cd858fe5d880 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:10.520803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1162/2334] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:27:10 2022] POST /identity/v3/users => generated 302 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:27:10.524620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d75ae-4f43-49e4-8f1f-7123026ee37e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:10.535894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d75ae-4f43-49e4-8f1f-7123026ee37e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:10.541326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12035cc-47c1-431a-bd77-863129cfd08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:10.541934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12035cc-47c1-431a-bd77-863129cfd08b admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:10.542262 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12035cc-47c1-431a-bd77-863129cfd08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:10.542537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12035cc-47c1-431a-bd77-863129cfd08b admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:10.543234 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12035cc-47c1-431a-bd77-863129cfd08b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:10.544831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12035cc-47c1-431a-bd77-863129cfd08b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:10.558401 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1173/2335] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:27:10.562728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c69569-b439-43ac-b786-cd858fe5d880 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:10.572862 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c69569-b439-43ac-b786-cd858fe5d880 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:10.577682 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038591e9-1364-407e-bd4e-7706a93c1608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:10.578449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038591e9-1364-407e-bd4e-7706a93c1608 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:10.578796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038591e9-1364-407e-bd4e-7706a93c1608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:10.579134 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038591e9-1364-407e-bd4e-7706a93c1608 admin admin] PATH_INFO: `/v3/projects/0f1211c1da7b44d2b1aa1407f4bd0d8e/users/9343ebc68ccb4bb5aeb104fb9d46306c/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:10.594587 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038591e9-1364-407e-bd4e-7706a93c1608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f1211c1da7b44d2b1aa1407f4bd0d8e, user_id=9343ebc68ccb4bb5aeb104fb9d46306c, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:10.597357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038591e9-1364-407e-bd4e-7706a93c1608 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:10.623160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1163/2336] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:27:10 2022] PUT /identity/v3/projects/0f1211c1da7b44d2b1aa1407f4bd0d8e/users/9343ebc68ccb4bb5aeb104fb9d46306c/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:27:10.626809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12035cc-47c1-431a-bd77-863129cfd08b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:10.639017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12035cc-47c1-431a-bd77-863129cfd08b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:10.717796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e2fcc7-95e6-4378-8d9d-a020f2c0fb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:10.718438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2fcc7-95e6-4378-8d9d-a020f2c0fb11 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:10.718717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2fcc7-95e6-4378-8d9d-a020f2c0fb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:10.718991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e2fcc7-95e6-4378-8d9d-a020f2c0fb11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:10.719640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2fcc7-95e6-4378-8d9d-a020f2c0fb11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:10.721295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e2fcc7-95e6-4378-8d9d-a020f2c0fb11 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:10.736254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1174/2337] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:27:10.737413 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038591e9-1364-407e-bd4e-7706a93c1608 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:10.743574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:10.749015 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038591e9-1364-407e-bd4e-7706a93c1608 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:10.755098 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c6e94-ef10-43d4-8f12-7b3603362827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:10.755759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c6e94-ef10-43d4-8f12-7b3603362827 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:10.756037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c6e94-ef10-43d4-8f12-7b3603362827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:10.756308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c6e94-ef10-43d4-8f12-7b3603362827 admin admin] PATH_INFO: `/v3/projects/0f1211c1da7b44d2b1aa1407f4bd0d8e/users/9343ebc68ccb4bb5aeb104fb9d46306c/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:10.758290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c6e94-ef10-43d4-8f12-7b3603362827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f1211c1da7b44d2b1aa1407f4bd0d8e, user_id=9343ebc68ccb4bb5aeb104fb9d46306c, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:10.761494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c6e94-ef10-43d4-8f12-7b3603362827 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:10.772385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1164/2338] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:27:10 2022] PUT /identity/v3/projects/0f1211c1da7b44d2b1aa1407f4bd0d8e/users/9343ebc68ccb4bb5aeb104fb9d46306c/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:10.783679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e5e3f-ec18-4815-b891-64e5085647bc None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:10.783963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e5e3f-ec18-4815-b891-64e5085647bc None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:10.784238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e5e3f-ec18-4815-b891-64e5085647bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:10.820599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c84e5e3f-ec18-4815-b891-64e5085647bc None None] MFA Rules not processed for user `9343ebc68ccb4bb5aeb104fb9d46306c`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:10.821964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c84e5e3f-ec18-4815-b891-64e5085647bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:10.913364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:10.940716 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1175/2339] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:10 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:10.956824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c6e94-ef10-43d4-8f12-7b3603362827 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:10.959204 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4e9c6e94-ef10-43d4-8f12-7b3603362827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:11.037922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c6e94-ef10-43d4-8f12-7b3603362827 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:11.043838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2fa47-86b9-4097-b3d1-a00b10255676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:11.044645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2fa47-86b9-4097-b3d1-a00b10255676 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:11.044959 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2fa47-86b9-4097-b3d1-a00b10255676 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:11.045232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2fa47-86b9-4097-b3d1-a00b10255676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:11.056137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2fa47-86b9-4097-b3d1-a00b10255676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:11.062273 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2fa47-86b9-4097-b3d1-a00b10255676 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:11.074137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1165/2340] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:27:10 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:27:11.126135 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e5e3f-ec18-4815-b891-64e5085647bc None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:11.127596 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c84e5e3f-ec18-4815-b891-64e5085647bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:11.144036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84e5e3f-ec18-4815-b891-64e5085647bc None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:11.147699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb4621-4105-4c0d-9d77-a888cc891735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:11.148306 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb4621-4105-4c0d-9d77-a888cc891735 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:11.148586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb4621-4105-4c0d-9d77-a888cc891735 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:11.148856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb4621-4105-4c0d-9d77-a888cc891735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:11.209034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb4621-4105-4c0d-9d77-a888cc891735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:11.210592 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb4621-4105-4c0d-9d77-a888cc891735 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:11.228067 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1176/2341] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:27:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:27:20.206480 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2fa47-86b9-4097-b3d1-a00b10255676 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:20.207935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ede2fa47-86b9-4097-b3d1-a00b10255676 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:20.285771 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2fa47-86b9-4097-b3d1-a00b10255676 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:20.291615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd313b64-4ddf-46ba-ad34-d540765bc05f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:20.293449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd313b64-4ddf-46ba-ad34-d540765bc05f service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:20.293793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd313b64-4ddf-46ba-ad34-d540765bc05f service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:20.294088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd313b64-4ddf-46ba-ad34-d540765bc05f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:20.295844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dd313b64-4ddf-46ba-ad34-d540765bc05f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:20.308072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd313b64-4ddf-46ba-ad34-d540765bc05f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:20.309645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd313b64-4ddf-46ba-ad34-d540765bc05f service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:20.323573 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1166/2342] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:27:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:27:21.179031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb4621-4105-4c0d-9d77-a888cc891735 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:21.252672 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb4621-4105-4c0d-9d77-a888cc891735 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:21.257490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:21.258219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:21.258590 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:21.258796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] PATH_INFO: `/v3/users/02a238f1347c42719b02611996730b02` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:21.260487 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02a238f1347c42719b02611996730b02)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:21.262468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:21.320379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02a238f1347c42719b02611996730b02'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:21.329527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02a238f1347c42719b02611996730b02'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:21.330356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02a238f1347c42719b02611996730b02'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:21.339977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02a238f1347c42719b02611996730b02'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:21.356222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02a238f1347c42719b02611996730b02'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:21.363274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1177/2343] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:27:21 2022] DELETE /identity/v3/users/02a238f1347c42719b02611996730b02 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:27:21.374580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ead55-40ce-4e1d-a8c8-15a7f8790063 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:21.375149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ead55-40ce-4e1d-a8c8-15a7f8790063 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:21.375149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ead55-40ce-4e1d-a8c8-15a7f8790063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:21.396494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d95ead55-40ce-4e1d-a8c8-15a7f8790063 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:21.397707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d95ead55-40ce-4e1d-a8c8-15a7f8790063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:21.467331 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1167/2344] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:21.473995 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:21.475531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:21.486065 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:21.567644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072356e1-2153-4d0c-abc1-93c4f23baca7 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:21.577886 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57336578-ad20-4a1c-af06-7984aed405d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:21.578479 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57336578-ad20-4a1c-af06-7984aed405d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:21.578845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57336578-ad20-4a1c-af06-7984aed405d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:21.579204 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57336578-ad20-4a1c-af06-7984aed405d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:21.589813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57336578-ad20-4a1c-af06-7984aed405d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:21.591272 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57336578-ad20-4a1c-af06-7984aed405d8 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:21.602861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1178/2345] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:27:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:27:22.097746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ead55-40ce-4e1d-a8c8-15a7f8790063 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:22.098942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d95ead55-40ce-4e1d-a8c8-15a7f8790063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:22.132211 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ead55-40ce-4e1d-a8c8-15a7f8790063 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:22.139825 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:22.140808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:22.141295 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:22.142095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] PATH_INFO: `/v3/projects/ac375b425c5e4fffb3c380a738988795` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:22.143383 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac375b425c5e4fffb3c380a738988795)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:22.146205 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:22.169946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:22.204017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac375b425c5e4fffb3c380a738988795', 'request_id': 'req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.209872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac375b425c5e4fffb3c380a738988795', 'request_id': 'req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.217340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac375b425c5e4fffb3c380a738988795', 'request_id': 'req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.228869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] The token cache is being invalidate because project ac375b425c5e4fffb3c380a738988795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:27:22.229148 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.229760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.231630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1168/2346] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:27:22 2022] DELETE /identity/v3/projects/ac375b425c5e4fffb3c380a738988795 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:22.235786 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57336578-ad20-4a1c-af06-7984aed405d8 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:22.245780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57336578-ad20-4a1c-af06-7984aed405d8 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:22.305228 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6178693-585e-4a31-8338-6d60d43b1562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:22.305852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6178693-585e-4a31-8338-6d60d43b1562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:22.306171 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6178693-585e-4a31-8338-6d60d43b1562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:22.306480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6178693-585e-4a31-8338-6d60d43b1562 admin admin] PATH_INFO: `/v3/users/9343ebc68ccb4bb5aeb104fb9d46306c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:22.307861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6178693-585e-4a31-8338-6d60d43b1562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9343ebc68ccb4bb5aeb104fb9d46306c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:22.309760 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6178693-585e-4a31-8338-6d60d43b1562 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:22.356712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c6178693-585e-4a31-8338-6d60d43b1562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9343ebc68ccb4bb5aeb104fb9d46306c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.363840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c6178693-585e-4a31-8338-6d60d43b1562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9343ebc68ccb4bb5aeb104fb9d46306c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.364507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c6178693-585e-4a31-8338-6d60d43b1562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9343ebc68ccb4bb5aeb104fb9d46306c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.374600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c6178693-585e-4a31-8338-6d60d43b1562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9343ebc68ccb4bb5aeb104fb9d46306c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.388844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c6178693-585e-4a31-8338-6d60d43b1562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9343ebc68ccb4bb5aeb104fb9d46306c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.394697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1179/2347] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:27:22 2022] DELETE /identity/v3/users/9343ebc68ccb4bb5aeb104fb9d46306c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:22.663821 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:22.665895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:22.678969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18c474-11b5-4df2-9f88-f16e37e5cb7b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:22.767146 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24514aa-8be5-46fc-ac23-1993c9e159ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:22.767914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24514aa-8be5-46fc-ac23-1993c9e159ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:22.768266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24514aa-8be5-46fc-ac23-1993c9e159ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:22.768615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24514aa-8be5-46fc-ac23-1993c9e159ff admin admin] PATH_INFO: `/v3/projects/0f1211c1da7b44d2b1aa1407f4bd0d8e` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:22.769872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24514aa-8be5-46fc-ac23-1993c9e159ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f1211c1da7b44d2b1aa1407f4bd0d8e)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:22.772043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24514aa-8be5-46fc-ac23-1993c9e159ff admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:22.830949 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f24514aa-8be5-46fc-ac23-1993c9e159ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f1211c1da7b44d2b1aa1407f4bd0d8e', 'request_id': 'req-f24514aa-8be5-46fc-ac23-1993c9e159ff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.836931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f24514aa-8be5-46fc-ac23-1993c9e159ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f1211c1da7b44d2b1aa1407f4bd0d8e', 'request_id': 'req-f24514aa-8be5-46fc-ac23-1993c9e159ff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.844299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f24514aa-8be5-46fc-ac23-1993c9e159ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f1211c1da7b44d2b1aa1407f4bd0d8e', 'request_id': 'req-f24514aa-8be5-46fc-ac23-1993c9e159ff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.857955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f24514aa-8be5-46fc-ac23-1993c9e159ff admin admin] The token cache is being invalidate because project 0f1211c1da7b44d2b1aa1407f4bd0d8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:27:22.858354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f24514aa-8be5-46fc-ac23-1993c9e159ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.859253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f24514aa-8be5-46fc-ac23-1993c9e159ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:22.860745 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1169/2348] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:27:22 2022] DELETE /identity/v3/projects/0f1211c1da7b44d2b1aa1407f4bd0d8e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:22.865909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b6366-8076-49cb-a5f1-74b08e956646 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:22.866298 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b6366-8076-49cb-a5f1-74b08e956646 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:22.866638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b6366-8076-49cb-a5f1-74b08e956646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:22.888587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-168b6366-8076-49cb-a5f1-74b08e956646 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:22.889874 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-168b6366-8076-49cb-a5f1-74b08e956646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:22.916677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:22.958127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1180/2349] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:22.967717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b8497-4987-47f0-8432-02f32e1761b9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:22.968072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b8497-4987-47f0-8432-02f32e1761b9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:22.968925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b8497-4987-47f0-8432-02f32e1761b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:22.994291 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-165b8497-4987-47f0-8432-02f32e1761b9 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:22.994291 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-165b8497-4987-47f0-8432-02f32e1761b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:23.002168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1170/2350] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:23.006481 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b6366-8076-49cb-a5f1-74b08e956646 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:23.017240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b6366-8076-49cb-a5f1-74b08e956646 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:23.021780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c9950-3d59-455f-a4ee-8ad67ee39b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:23.022407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c9950-3d59-455f-a4ee-8ad67ee39b00 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:23.022683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c9950-3d59-455f-a4ee-8ad67ee39b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:23.022950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c9950-3d59-455f-a4ee-8ad67ee39b00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:23.023594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c9950-3d59-455f-a4ee-8ad67ee39b00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:27:23.023893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c9950-3d59-455f-a4ee-8ad67ee39b00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:23.026050 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c9950-3d59-455f-a4ee-8ad67ee39b00 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:23.037446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1181/2351] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:27:23.041950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b8497-4987-47f0-8432-02f32e1761b9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:23.051890 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b8497-4987-47f0-8432-02f32e1761b9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:23.056522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbbb83-2efb-4aa3-9e17-571eeb974585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:23.057140 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbbb83-2efb-4aa3-9e17-571eeb974585 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:23.057441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbbb83-2efb-4aa3-9e17-571eeb974585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:23.057713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dbbb83-2efb-4aa3-9e17-571eeb974585 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:23.058391 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbbb83-2efb-4aa3-9e17-571eeb974585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:23.060261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dbbb83-2efb-4aa3-9e17-571eeb974585 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:23.089575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1171/2352] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:27:23 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:27:23.093059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c9950-3d59-455f-a4ee-8ad67ee39b00 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:23.106005 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c9950-3d59-455f-a4ee-8ad67ee39b00 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:23.177982 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369ceb3-fb9c-4552-8f49-faf98b903ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:23.178773 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369ceb3-fb9c-4552-8f49-faf98b903ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:23.179124 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369ceb3-fb9c-4552-8f49-faf98b903ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:23.179509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369ceb3-fb9c-4552-8f49-faf98b903ed6 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:23.180226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369ceb3-fb9c-4552-8f49-faf98b903ed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:23.182527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369ceb3-fb9c-4552-8f49-faf98b903ed6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:23.222012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1182/2353] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:27:23 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:27:23.223421 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbbb83-2efb-4aa3-9e17-571eeb974585 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:23.235971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dbbb83-2efb-4aa3-9e17-571eeb974585 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:23.240794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dfa5c-8b57-4cf3-833b-1c2664082783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:23.241524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dfa5c-8b57-4cf3-833b-1c2664082783 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:23.241661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dfa5c-8b57-4cf3-833b-1c2664082783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:23.241937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dfa5c-8b57-4cf3-833b-1c2664082783 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:23.242759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197dfa5c-8b57-4cf3-833b-1c2664082783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:23.244600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197dfa5c-8b57-4cf3-833b-1c2664082783 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:23.246389 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:23.262826 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1172/2354] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:27:23.263829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369ceb3-fb9c-4552-8f49-faf98b903ed6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:23.284262 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369ceb3-fb9c-4552-8f49-faf98b903ed6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:23.288887 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49532324-9d83-4692-b1fe-2c2e64ecb45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:23.289469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49532324-9d83-4692-b1fe-2c2e64ecb45e admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:23.289746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49532324-9d83-4692-b1fe-2c2e64ecb45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:23.290064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49532324-9d83-4692-b1fe-2c2e64ecb45e admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:23.290844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49532324-9d83-4692-b1fe-2c2e64ecb45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:23.292422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49532324-9d83-4692-b1fe-2c2e64ecb45e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:23.304614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1183/2355] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:27:23.309613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dfa5c-8b57-4cf3-833b-1c2664082783 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:23.322774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dfa5c-8b57-4cf3-833b-1c2664082783 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:23.327104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f66e8-aac8-49fb-882d-79dea89b41ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:23.327836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f66e8-aac8-49fb-882d-79dea89b41ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:23.328217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f66e8-aac8-49fb-882d-79dea89b41ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:23.328673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f66e8-aac8-49fb-882d-79dea89b41ed admin admin] PATH_INFO: `/v3/projects/b87578571d6e436980a3f94ccefc9342/users/2783e95a7d5344eaa311624083bf7abb/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:23.344755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f66e8-aac8-49fb-882d-79dea89b41ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=b87578571d6e436980a3f94ccefc9342, user_id=2783e95a7d5344eaa311624083bf7abb, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:23.347832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f66e8-aac8-49fb-882d-79dea89b41ed admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:23.364641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1173/2356] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:27:23 2022] PUT /identity/v3/projects/b87578571d6e436980a3f94ccefc9342/users/2783e95a7d5344eaa311624083bf7abb/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:23.376850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd3079-7749-4d82-942a-99298cef789e None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:23.377135 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd3079-7749-4d82-942a-99298cef789e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:23.377435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd3079-7749-4d82-942a-99298cef789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:23.410852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f7fd3079-7749-4d82-942a-99298cef789e None None] MFA Rules not processed for user `2783e95a7d5344eaa311624083bf7abb`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:23.412148 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f7fd3079-7749-4d82-942a-99298cef789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:23.480258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1184/2357] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:23.491580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f66e8-aac8-49fb-882d-79dea89b41ed admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:23.493296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9f6f66e8-aac8-49fb-882d-79dea89b41ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:23.549809 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f66e8-aac8-49fb-882d-79dea89b41ed admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:23.553529 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5ab97-d66b-4f7f-908a-28c97a903726 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:23.554160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5ab97-d66b-4f7f-908a-28c97a903726 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:23.554439 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5ab97-d66b-4f7f-908a-28c97a903726 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:23.554712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5ab97-d66b-4f7f-908a-28c97a903726 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:23.563590 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5ab97-d66b-4f7f-908a-28c97a903726 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:23.565013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5ab97-d66b-4f7f-908a-28c97a903726 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:23.573900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1174/2358] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:27:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:27:23.594889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd3079-7749-4d82-942a-99298cef789e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:23.627316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:23.675817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd3079-7749-4d82-942a-99298cef789e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:23.681906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:23.682734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:23.683013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:23.683286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] PATH_INFO: `/v3/users/2783e95a7d5344eaa311624083bf7abb` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:23.684425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2783e95a7d5344eaa311624083bf7abb)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:23.686379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:23.734423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2783e95a7d5344eaa311624083bf7abb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:23.741216 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2783e95a7d5344eaa311624083bf7abb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:23.741884 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2783e95a7d5344eaa311624083bf7abb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:23.750220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2783e95a7d5344eaa311624083bf7abb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:23.762509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2783e95a7d5344eaa311624083bf7abb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:23.768428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1185/2359] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:27:23 2022] DELETE /identity/v3/users/2783e95a7d5344eaa311624083bf7abb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:23.772842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039a021-1597-44da-a4b4-6fa7a6faface None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:23.773303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039a021-1597-44da-a4b4-6fa7a6faface None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:23.773512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039a021-1597-44da-a4b4-6fa7a6faface None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:23.792661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e039a021-1597-44da-a4b4-6fa7a6faface None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:23.793861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e039a021-1597-44da-a4b4-6fa7a6faface None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:23.853375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1175/2360] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:23.863349 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:23.864784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:23.923467 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47ec39-de01-4ae6-979d-280db26e648c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:23.927051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe5802-77c5-4c19-b91a-45b8fdc6d977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:23.927658 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe5802-77c5-4c19-b91a-45b8fdc6d977 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:23.928001 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe5802-77c5-4c19-b91a-45b8fdc6d977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:23.928186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe5802-77c5-4c19-b91a-45b8fdc6d977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:23.937620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe5802-77c5-4c19-b91a-45b8fdc6d977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:23.939079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe5802-77c5-4c19-b91a-45b8fdc6d977 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:23.949165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1186/2361] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:27:24.237933 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039a021-1597-44da-a4b4-6fa7a6faface None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:24.239493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e039a021-1597-44da-a4b4-6fa7a6faface None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:24.255780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039a021-1597-44da-a4b4-6fa7a6faface None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:24.261470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47462632-2afa-4072-b980-71aacff7a630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:24.262162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47462632-2afa-4072-b980-71aacff7a630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:24.262448 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47462632-2afa-4072-b980-71aacff7a630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:24.262783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47462632-2afa-4072-b980-71aacff7a630 admin admin] PATH_INFO: `/v3/projects/b87578571d6e436980a3f94ccefc9342` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:24.263837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47462632-2afa-4072-b980-71aacff7a630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b87578571d6e436980a3f94ccefc9342)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:24.265760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47462632-2afa-4072-b980-71aacff7a630 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:24.317546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-47462632-2afa-4072-b980-71aacff7a630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b87578571d6e436980a3f94ccefc9342', 'request_id': 'req-47462632-2afa-4072-b980-71aacff7a630'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:24.324621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-47462632-2afa-4072-b980-71aacff7a630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b87578571d6e436980a3f94ccefc9342', 'request_id': 'req-47462632-2afa-4072-b980-71aacff7a630'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:24.331143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-47462632-2afa-4072-b980-71aacff7a630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b87578571d6e436980a3f94ccefc9342', 'request_id': 'req-47462632-2afa-4072-b980-71aacff7a630'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:24.340577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:24.346603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-47462632-2afa-4072-b980-71aacff7a630 admin admin] The token cache is being invalidate because project b87578571d6e436980a3f94ccefc9342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:27:24.347144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-47462632-2afa-4072-b980-71aacff7a630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:24.348343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-47462632-2afa-4072-b980-71aacff7a630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:27:24.350181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1176/2362] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:27:24 2022] DELETE /identity/v3/projects/b87578571d6e436980a3f94ccefc9342 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:27:24.355089 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe14083-11f6-430f-b63f-d5e3e8a1f391 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:24.355822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe14083-11f6-430f-b63f-d5e3e8a1f391 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:24.355925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe14083-11f6-430f-b63f-d5e3e8a1f391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:24.376416 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-fbe14083-11f6-430f-b63f-d5e3e8a1f391 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:24.377611 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fbe14083-11f6-430f-b63f-d5e3e8a1f391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:24.419990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:24.438364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1187/2363] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:24.445891 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac850c3-bfed-4cdb-b164-f07c692110c4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:24.446196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac850c3-bfed-4cdb-b164-f07c692110c4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:24.446961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac850c3-bfed-4cdb-b164-f07c692110c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:24.467842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2ac850c3-bfed-4cdb-b164-f07c692110c4 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:24.469921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2ac850c3-bfed-4cdb-b164-f07c692110c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:24.479622 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1177/2364] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:24.481820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe14083-11f6-430f-b63f-d5e3e8a1f391 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:24.492007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe14083-11f6-430f-b63f-d5e3e8a1f391 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:24.497284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de6299-f85a-42c2-a43c-000a732df365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:24.497962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de6299-f85a-42c2-a43c-000a732df365 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:24.498334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de6299-f85a-42c2-a43c-000a732df365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:24.498715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de6299-f85a-42c2-a43c-000a732df365 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:24.499443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de6299-f85a-42c2-a43c-000a732df365 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:27:24.499815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de6299-f85a-42c2-a43c-000a732df365 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:24.501499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de6299-f85a-42c2-a43c-000a732df365 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:24.514664 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1188/2365] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:27:24.519280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac850c3-bfed-4cdb-b164-f07c692110c4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:24.529097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac850c3-bfed-4cdb-b164-f07c692110c4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:24.533711 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84963b-e39e-477b-b6e1-9d8e4334ee95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:24.534362 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84963b-e39e-477b-b6e1-9d8e4334ee95 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:24.534635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84963b-e39e-477b-b6e1-9d8e4334ee95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:24.534899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84963b-e39e-477b-b6e1-9d8e4334ee95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:24.535558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84963b-e39e-477b-b6e1-9d8e4334ee95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:24.537429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84963b-e39e-477b-b6e1-9d8e4334ee95 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:24.568084 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1178/2366] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:27:24 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:27:24.569265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de6299-f85a-42c2-a43c-000a732df365 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:24.579777 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de6299-f85a-42c2-a43c-000a732df365 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:24.640535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897ce1b-100d-44bd-89c0-dc9d18d32273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:24.641142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897ce1b-100d-44bd-89c0-dc9d18d32273 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:24.641422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897ce1b-100d-44bd-89c0-dc9d18d32273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:24.641696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897ce1b-100d-44bd-89c0-dc9d18d32273 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:24.642378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897ce1b-100d-44bd-89c0-dc9d18d32273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:24.644296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897ce1b-100d-44bd-89c0-dc9d18d32273 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:24.670125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1189/2367] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:27:24 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:27:24.671305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84963b-e39e-477b-b6e1-9d8e4334ee95 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:24.682922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84963b-e39e-477b-b6e1-9d8e4334ee95 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:24.687496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea63441-d38b-4199-9b63-a6c5179dca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:24.688083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea63441-d38b-4199-9b63-a6c5179dca71 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:24.688352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea63441-d38b-4199-9b63-a6c5179dca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:24.688669 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea63441-d38b-4199-9b63-a6c5179dca71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:24.689246 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea63441-d38b-4199-9b63-a6c5179dca71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:24.690867 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea63441-d38b-4199-9b63-a6c5179dca71 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:24.701030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1179/2368] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:27:24.705283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897ce1b-100d-44bd-89c0-dc9d18d32273 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:24.716153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897ce1b-100d-44bd-89c0-dc9d18d32273 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:24.720990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c49dc-9a61-4892-9b7b-2565cd1bc149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:24.721681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c49dc-9a61-4892-9b7b-2565cd1bc149 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:24.722053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c49dc-9a61-4892-9b7b-2565cd1bc149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:24.722421 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c49dc-9a61-4892-9b7b-2565cd1bc149 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:24.723139 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c49dc-9a61-4892-9b7b-2565cd1bc149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:24.724808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1c49dc-9a61-4892-9b7b-2565cd1bc149 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:24.735891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1190/2369] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:27:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:27:24.739957 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea63441-d38b-4199-9b63-a6c5179dca71 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:24.750716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea63441-d38b-4199-9b63-a6c5179dca71 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:24.755535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1f9c0-ab37-426e-9ed4-b44a8e449582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:24.756180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1f9c0-ab37-426e-9ed4-b44a8e449582 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:24.756453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1f9c0-ab37-426e-9ed4-b44a8e449582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:24.756721 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1f9c0-ab37-426e-9ed4-b44a8e449582 admin admin] PATH_INFO: `/v3/projects/3d0576112faf4d5c91ddacb0668b4e5f/users/3d153c3c6eb64712b7ec657e0ccb34fb/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:24.772169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1f9c0-ab37-426e-9ed4-b44a8e449582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d0576112faf4d5c91ddacb0668b4e5f, user_id=3d153c3c6eb64712b7ec657e0ccb34fb, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:24.775004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1f9c0-ab37-426e-9ed4-b44a8e449582 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:24.792541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1180/2370] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:27:24 2022] PUT /identity/v3/projects/3d0576112faf4d5c91ddacb0668b4e5f/users/3d153c3c6eb64712b7ec657e0ccb34fb/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:27:24.861729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacca49-9900-438e-b47c-915c2fa03ca2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:24.862005 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacca49-9900-438e-b47c-915c2fa03ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:24.862351 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacca49-9900-438e-b47c-915c2fa03ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:24.890651 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3eacca49-9900-438e-b47c-915c2fa03ca2 None None] MFA Rules not processed for user `3d153c3c6eb64712b7ec657e0ccb34fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:27:24.891851 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3eacca49-9900-438e-b47c-915c2fa03ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:24.959571 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1191/2371] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:27:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:27:24.970491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1f9c0-ab37-426e-9ed4-b44a8e449582 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:27:24.972442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1cb1f9c0-ab37-426e-9ed4-b44a8e449582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:27:25.032329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1f9c0-ab37-426e-9ed4-b44a8e449582 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:27:25.038537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f0c84-f227-49f0-bd02-cac1b5090b06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:27:25.041900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f0c84-f227-49f0-bd02-cac1b5090b06 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:27:25.042324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f0c84-f227-49f0-bd02-cac1b5090b06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:27:25.042728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f0c84-f227-49f0-bd02-cac1b5090b06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:27:25.058324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f0c84-f227-49f0-bd02-cac1b5090b06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:27:25.060064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f0c84-f227-49f0-bd02-cac1b5090b06 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:27:25.070835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1181/2372] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:27:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:27:34.956141 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:27:35.065294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:28:03.098224 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacca49-9900-438e-b47c-915c2fa03ca2 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:03.167107 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacca49-9900-438e-b47c-915c2fa03ca2 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:03.171671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:03.172299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:03.172579 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:03.172850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] PATH_INFO: `/v3/users/3d153c3c6eb64712b7ec657e0ccb34fb` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:03.173970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d153c3c6eb64712b7ec657e0ccb34fb)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:03.175904 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:03.228384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d153c3c6eb64712b7ec657e0ccb34fb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:03.235734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d153c3c6eb64712b7ec657e0ccb34fb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:03.236438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d153c3c6eb64712b7ec657e0ccb34fb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:03.245075 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d153c3c6eb64712b7ec657e0ccb34fb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:03.264634 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d153c3c6eb64712b7ec657e0ccb34fb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:03.271182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1192/2373] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:28:03 2022] DELETE /identity/v3/users/3d153c3c6eb64712b7ec657e0ccb34fb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:28:03.275637 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aee99b-4d11-451f-9ed3-d8660c06b578 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:03.276128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aee99b-4d11-451f-9ed3-d8660c06b578 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:03.276227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aee99b-4d11-451f-9ed3-d8660c06b578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:03.296263 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-16aee99b-4d11-451f-9ed3-d8660c06b578 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:28:03.297446 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-16aee99b-4d11-451f-9ed3-d8660c06b578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:03.361464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1182/2374] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:28:03.371891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:03.373294 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:03.437397 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432db8ec-96a7-4800-8d4f-7aa720de6da4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:03.441376 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51827ae2-aa9b-4477-a648-225c2a4c3d9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:03.441983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51827ae2-aa9b-4477-a648-225c2a4c3d9b service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:03.442291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51827ae2-aa9b-4477-a648-225c2a4c3d9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:03.442564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51827ae2-aa9b-4477-a648-225c2a4c3d9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:03.454048 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51827ae2-aa9b-4477-a648-225c2a4c3d9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:03.455529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51827ae2-aa9b-4477-a648-225c2a4c3d9b service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:03.465271 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1193/2375] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:28:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:28:03.739526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aee99b-4d11-451f-9ed3-d8660c06b578 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:03.739526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-16aee99b-4d11-451f-9ed3-d8660c06b578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:03.751092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aee99b-4d11-451f-9ed3-d8660c06b578 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:03.756088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:03.756915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:03.757226 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:03.757495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] PATH_INFO: `/v3/projects/3d0576112faf4d5c91ddacb0668b4e5f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:03.758672 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d0576112faf4d5c91ddacb0668b4e5f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:03.760686 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:03.833348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d0576112faf4d5c91ddacb0668b4e5f', 'request_id': 'req-8627065c-1afa-4570-98df-f6c47b3d4649'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:03.840151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d0576112faf4d5c91ddacb0668b4e5f', 'request_id': 'req-8627065c-1afa-4570-98df-f6c47b3d4649'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:03.846933 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d0576112faf4d5c91ddacb0668b4e5f', 'request_id': 'req-8627065c-1afa-4570-98df-f6c47b3d4649'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:03.866694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] The token cache is being invalidate because project 3d0576112faf4d5c91ddacb0668b4e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:28:03.866997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:03.867709 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:03.870348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1183/2376] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:28:03 2022] DELETE /identity/v3/projects/3d0576112faf4d5c91ddacb0668b4e5f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:28:13.460931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:28:13.862457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:28:23.038217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d425de8-2c0b-475c-8ebd-70645d200157 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:23.038770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d425de8-2c0b-475c-8ebd-70645d200157 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:23.038770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d425de8-2c0b-475c-8ebd-70645d200157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:23.058286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2d425de8-2c0b-475c-8ebd-70645d200157 None None] MFA Rules not processed for user `1dba953d778c418abea22efac408e2db`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:28:23.059469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2d425de8-2c0b-475c-8ebd-70645d200157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:23.113150 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1194/2377] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:28:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:28:25.387829 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:25.389798 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:25.461387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627065c-1afa-4570-98df-f6c47b3d4649 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:25.465063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5d532-e1d9-40fa-a4d0-79d364bf7542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:25.465772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5d532-e1d9-40fa-a4d0-79d364bf7542 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:25.466345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5d532-e1d9-40fa-a4d0-79d364bf7542 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:25.466712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5d532-e1d9-40fa-a4d0-79d364bf7542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:25.468518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e8a5d532-e1d9-40fa-a4d0-79d364bf7542 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:25.544085 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5d532-e1d9-40fa-a4d0-79d364bf7542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:25.545718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5d532-e1d9-40fa-a4d0-79d364bf7542 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:25.555871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1184/2378] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:28:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:28:30.383448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d425de8-2c0b-475c-8ebd-70645d200157 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:30.384959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2d425de8-2c0b-475c-8ebd-70645d200157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:30.448508 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d425de8-2c0b-475c-8ebd-70645d200157 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:30.452888 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:30.453501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:30.453771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:30.454037 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] PATH_INFO: `/v3/users/7d1bc9100d5a4d1fa28bbcac495fbfa7` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:30.455153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d1bc9100d5a4d1fa28bbcac495fbfa7)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:30.457053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:30.503011 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d1bc9100d5a4d1fa28bbcac495fbfa7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:30.509991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d1bc9100d5a4d1fa28bbcac495fbfa7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:30.510670 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d1bc9100d5a4d1fa28bbcac495fbfa7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:30.518016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d1bc9100d5a4d1fa28bbcac495fbfa7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:30.531437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d1bc9100d5a4d1fa28bbcac495fbfa7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:30.537515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1195/2379] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:28:30 2022] DELETE /identity/v3/users/7d1bc9100d5a4d1fa28bbcac495fbfa7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:28:30.542704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd464ee-518f-4c09-b171-df991b7557e1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:30.543160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd464ee-518f-4c09-b171-df991b7557e1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:30.543256 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd464ee-518f-4c09-b171-df991b7557e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:30.562267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ffd464ee-518f-4c09-b171-df991b7557e1 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:28:30.563464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ffd464ee-518f-4c09-b171-df991b7557e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:30.621158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1185/2380] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:28:30.630734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:30.632363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:30.660275 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:28:30.692117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020be0b-0e08-49ae-a255-28beaa0f68ce admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:30.695858 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554db48c-2bb8-4b33-aedb-50de471a0060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:30.696439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554db48c-2bb8-4b33-aedb-50de471a0060 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:30.696842 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554db48c-2bb8-4b33-aedb-50de471a0060 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:30.697121 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554db48c-2bb8-4b33-aedb-50de471a0060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:30.706244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554db48c-2bb8-4b33-aedb-50de471a0060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:30.707697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554db48c-2bb8-4b33-aedb-50de471a0060 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:30.717957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1196/2381] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:28:31.071706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd464ee-518f-4c09-b171-df991b7557e1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:31.073687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ffd464ee-518f-4c09-b171-df991b7557e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:31.091119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd464ee-518f-4c09-b171-df991b7557e1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:31.096694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:31.097244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:31.097511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:31.097767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044 admin admin] PATH_INFO: `/v3/projects/438e2c1f93bd41a7bd517a950b06d6ff` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:31.098974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=438e2c1f93bd41a7bd517a950b06d6ff)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:31.100896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:31.145519 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:28:31.148983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '438e2c1f93bd41a7bd517a950b06d6ff', 'request_id': 'req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:31.155127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '438e2c1f93bd41a7bd517a950b06d6ff', 'request_id': 'req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:31.161311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '438e2c1f93bd41a7bd517a950b06d6ff', 'request_id': 'req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:31.174609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044 admin admin] The token cache is being invalidate because project 438e2c1f93bd41a7bd517a950b06d6ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:28:31.174902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:31.175528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1bf1ca5c-bd39-4da7-a408-6a8dba7b7044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:28:31.177034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1186/2382] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:28:31 2022] DELETE /identity/v3/projects/438e2c1f93bd41a7bd517a950b06d6ff => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:28:31.182255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902db53-c8ee-4756-8dfb-245398e48452 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:31.182532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902db53-c8ee-4756-8dfb-245398e48452 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:31.182799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902db53-c8ee-4756-8dfb-245398e48452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:31.212612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2902db53-c8ee-4756-8dfb-245398e48452 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:28:31.213796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2902db53-c8ee-4756-8dfb-245398e48452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:31.282983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1197/2383] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:28:31.290938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0ab15-890b-40d9-9f9c-9b7d630546ef None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:31.291223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0ab15-890b-40d9-9f9c-9b7d630546ef None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:31.291494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0ab15-890b-40d9-9f9c-9b7d630546ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:31.311632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c4f0ab15-890b-40d9-9f9c-9b7d630546ef None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:28:31.312869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c4f0ab15-890b-40d9-9f9c-9b7d630546ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:31.320810 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1187/2384] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:28:31.325645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902db53-c8ee-4756-8dfb-245398e48452 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:31.335306 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902db53-c8ee-4756-8dfb-245398e48452 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:31.340017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f92ed-68ef-4083-8e32-5b3b0255ee78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:31.340608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f92ed-68ef-4083-8e32-5b3b0255ee78 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:31.340883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f92ed-68ef-4083-8e32-5b3b0255ee78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:31.341149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f92ed-68ef-4083-8e32-5b3b0255ee78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:31.341791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f92ed-68ef-4083-8e32-5b3b0255ee78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:28:31.342115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f92ed-68ef-4083-8e32-5b3b0255ee78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:31.343699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f92ed-68ef-4083-8e32-5b3b0255ee78 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:31.354975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1198/2385] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:28:31.359391 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0ab15-890b-40d9-9f9c-9b7d630546ef None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:31.368445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0ab15-890b-40d9-9f9c-9b7d630546ef None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:31.373640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42caa0-b33e-4ee5-9892-79ee0020fe3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:31.374250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42caa0-b33e-4ee5-9892-79ee0020fe3b admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:31.374528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42caa0-b33e-4ee5-9892-79ee0020fe3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:31.374801 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b42caa0-b33e-4ee5-9892-79ee0020fe3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:31.375423 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42caa0-b33e-4ee5-9892-79ee0020fe3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:31.377305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b42caa0-b33e-4ee5-9892-79ee0020fe3b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:31.404556 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1188/2386] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:28:31 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:28:31.408295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f92ed-68ef-4083-8e32-5b3b0255ee78 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:31.419408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f92ed-68ef-4083-8e32-5b3b0255ee78 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:31.479202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee2123-e6aa-4bd1-a7e5-40f2e73a2f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:31.479804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee2123-e6aa-4bd1-a7e5-40f2e73a2f48 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:31.480078 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee2123-e6aa-4bd1-a7e5-40f2e73a2f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:31.480343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee2123-e6aa-4bd1-a7e5-40f2e73a2f48 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:31.481232 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee2123-e6aa-4bd1-a7e5-40f2e73a2f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:31.483189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee2123-e6aa-4bd1-a7e5-40f2e73a2f48 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:31.504547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1199/2387] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:28:31 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:28:31.508609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42caa0-b33e-4ee5-9892-79ee0020fe3b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:31.519688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b42caa0-b33e-4ee5-9892-79ee0020fe3b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:31.527791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6a451-ba05-4a3f-8d5c-0c77ea159ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:31.528531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6a451-ba05-4a3f-8d5c-0c77ea159ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:31.528875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6a451-ba05-4a3f-8d5c-0c77ea159ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:31.529215 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6a451-ba05-4a3f-8d5c-0c77ea159ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:31.529944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6a451-ba05-4a3f-8d5c-0c77ea159ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:31.531804 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6a451-ba05-4a3f-8d5c-0c77ea159ca7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:31.543086 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1189/2388] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:28:31.547899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee2123-e6aa-4bd1-a7e5-40f2e73a2f48 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:31.559887 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee2123-e6aa-4bd1-a7e5-40f2e73a2f48 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:31.564884 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e934090-9c00-4629-9f91-8fd6f080bb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:31.565494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e934090-9c00-4629-9f91-8fd6f080bb50 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:31.565769 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e934090-9c00-4629-9f91-8fd6f080bb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:31.566035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e934090-9c00-4629-9f91-8fd6f080bb50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:31.566795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e934090-9c00-4629-9f91-8fd6f080bb50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:31.568402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e934090-9c00-4629-9f91-8fd6f080bb50 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:31.595243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1200/2389] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:28:31.596649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6a451-ba05-4a3f-8d5c-0c77ea159ca7 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:31.608418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6a451-ba05-4a3f-8d5c-0c77ea159ca7 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:31.616986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b756b9-4720-41c8-9c7b-0fe9b1c4616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:31.618337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b756b9-4720-41c8-9c7b-0fe9b1c4616e admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:31.619309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b756b9-4720-41c8-9c7b-0fe9b1c4616e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:31.619768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b756b9-4720-41c8-9c7b-0fe9b1c4616e admin admin] PATH_INFO: `/v3/projects/890469995c02465eb9b9ad2d1fb9ab6f/users/fa5bf070765946168f7e5937f2b076c1/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:31.640873 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b756b9-4720-41c8-9c7b-0fe9b1c4616e admin admin] RBAC: Authorizing `identity:create_grant(project_id=890469995c02465eb9b9ad2d1fb9ab6f, user_id=fa5bf070765946168f7e5937f2b076c1, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:31.643842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b756b9-4720-41c8-9c7b-0fe9b1c4616e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:31.663329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1190/2390] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:28:31 2022] PUT /identity/v3/projects/890469995c02465eb9b9ad2d1fb9ab6f/users/fa5bf070765946168f7e5937f2b076c1/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:28:31.665125 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379cb5f-41a2-4c9e-ba00-8d6d398dfc81 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:31.665480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379cb5f-41a2-4c9e-ba00-8d6d398dfc81 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:31.666383 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379cb5f-41a2-4c9e-ba00-8d6d398dfc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:31.686927 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1379cb5f-41a2-4c9e-ba00-8d6d398dfc81 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:28:31.688157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1379cb5f-41a2-4c9e-ba00-8d6d398dfc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:31.719883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:28:31.758408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1201/2391] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:28:31.772760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b756b9-4720-41c8-9c7b-0fe9b1c4616e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:31.774778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-33b756b9-4720-41c8-9c7b-0fe9b1c4616e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:31.843667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b756b9-4720-41c8-9c7b-0fe9b1c4616e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:31.847615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf0e58-4253-43cf-8a1a-e8e999fc16b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:31.848296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf0e58-4253-43cf-8a1a-e8e999fc16b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:31.848645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf0e58-4253-43cf-8a1a-e8e999fc16b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:31.848985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf0e58-4253-43cf-8a1a-e8e999fc16b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:31.859360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf0e58-4253-43cf-8a1a-e8e999fc16b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:31.860926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf0e58-4253-43cf-8a1a-e8e999fc16b5 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:31.870801 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1191/2392] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:28:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:28:36.095539 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12a8d3-c113-438c-8566-4be1a31dddec None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:36.096060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12a8d3-c113-438c-8566-4be1a31dddec None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:36.096200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12a8d3-c113-438c-8566-4be1a31dddec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:36.126965 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2e12a8d3-c113-438c-8566-4be1a31dddec None None] MFA Rules not processed for user `fa5bf070765946168f7e5937f2b076c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:28:36.128171 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2e12a8d3-c113-438c-8566-4be1a31dddec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:36.201666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1202/2393] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:28:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:28:36.206523 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf0e58-4253-43cf-8a1a-e8e999fc16b5 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:36.217632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf0e58-4253-43cf-8a1a-e8e999fc16b5 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:36.222506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddee0f6-7dbd-497f-9a33-7c245da3177e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:36.223118 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddee0f6-7dbd-497f-9a33-7c245da3177e admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:36.223393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddee0f6-7dbd-497f-9a33-7c245da3177e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:36.223661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddee0f6-7dbd-497f-9a33-7c245da3177e admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:36.224292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddee0f6-7dbd-497f-9a33-7c245da3177e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:36.226236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddee0f6-7dbd-497f-9a33-7c245da3177e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:36.263904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1192/2394] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:28:36 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:28:36.265423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12a8d3-c113-438c-8566-4be1a31dddec None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:36.277235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12a8d3-c113-438c-8566-4be1a31dddec None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:36.354808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce8080-b907-4b99-9858-f96a7ad40df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:36.355622 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce8080-b907-4b99-9858-f96a7ad40df6 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:36.356155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce8080-b907-4b99-9858-f96a7ad40df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:36.356425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce8080-b907-4b99-9858-f96a7ad40df6 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:36.357058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce8080-b907-4b99-9858-f96a7ad40df6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:36.359091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce8080-b907-4b99-9858-f96a7ad40df6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:36.382505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1203/2395] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:28:36 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:28:36.386802 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddee0f6-7dbd-497f-9a33-7c245da3177e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:36.397223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddee0f6-7dbd-497f-9a33-7c245da3177e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:36.402497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f23b31-2821-4d6c-a35a-0bec63f56e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:36.403092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f23b31-2821-4d6c-a35a-0bec63f56e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:36.403368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f23b31-2821-4d6c-a35a-0bec63f56e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:36.403634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f23b31-2821-4d6c-a35a-0bec63f56e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:36.404266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f23b31-2821-4d6c-a35a-0bec63f56e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:36.405928 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f23b31-2821-4d6c-a35a-0bec63f56e5a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:36.421920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1193/2396] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:28:36.423246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce8080-b907-4b99-9858-f96a7ad40df6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:36.434034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce8080-b907-4b99-9858-f96a7ad40df6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:36.440545 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ad818-86d5-4258-8bdf-92518316533f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:36.441264 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ad818-86d5-4258-8bdf-92518316533f admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:36.441549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ad818-86d5-4258-8bdf-92518316533f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:36.441830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ad818-86d5-4258-8bdf-92518316533f admin admin] PATH_INFO: `/v3/projects/69f7ac71eb644876b683a468891ea5a7/users/264dabfe887e4866a07e9992cc8ba594/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:36.459679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ad818-86d5-4258-8bdf-92518316533f admin admin] RBAC: Authorizing `identity:create_grant(project_id=69f7ac71eb644876b683a468891ea5a7, user_id=264dabfe887e4866a07e9992cc8ba594, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:36.463154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ad818-86d5-4258-8bdf-92518316533f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:36.483329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1204/2397] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:28:36 2022] PUT /identity/v3/projects/69f7ac71eb644876b683a468891ea5a7/users/264dabfe887e4866a07e9992cc8ba594/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:28:36.485093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f23b31-2821-4d6c-a35a-0bec63f56e5a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:36.497187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f23b31-2821-4d6c-a35a-0bec63f56e5a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:36.538559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:28:36.560935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f0eed-e12b-422a-aed0-5f0a2bb00bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:36.561542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f0eed-e12b-422a-aed0-5f0a2bb00bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:36.561818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f0eed-e12b-422a-aed0-5f0a2bb00bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:36.562112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f0eed-e12b-422a-aed0-5f0a2bb00bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:36.562751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f0eed-e12b-422a-aed0-5f0a2bb00bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:36.564334 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f0eed-e12b-422a-aed0-5f0a2bb00bc6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:36.575448 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1194/2398] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:28:36.579430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ad818-86d5-4258-8bdf-92518316533f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:36.589825 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ad818-86d5-4258-8bdf-92518316533f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:36.594483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b197f-2595-49cf-9ead-19477f055a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:36.595174 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b197f-2595-49cf-9ead-19477f055a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:36.595474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b197f-2595-49cf-9ead-19477f055a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:36.595747 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b197f-2595-49cf-9ead-19477f055a17 admin admin] PATH_INFO: `/v3/projects/69f7ac71eb644876b683a468891ea5a7/users/264dabfe887e4866a07e9992cc8ba594/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:36.597730 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b197f-2595-49cf-9ead-19477f055a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69f7ac71eb644876b683a468891ea5a7, user_id=264dabfe887e4866a07e9992cc8ba594, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:36.600383 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b197f-2595-49cf-9ead-19477f055a17 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:36.611074 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1205/2399] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:28:36 2022] PUT /identity/v3/projects/69f7ac71eb644876b683a468891ea5a7/users/264dabfe887e4866a07e9992cc8ba594/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:28:42.001921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05bd9b-d836-48c4-8913-3bddff805df1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:42.002498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05bd9b-d836-48c4-8913-3bddff805df1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:42.002498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05bd9b-d836-48c4-8913-3bddff805df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:42.039919 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6d05bd9b-d836-48c4-8913-3bddff805df1 None None] MFA Rules not processed for user `264dabfe887e4866a07e9992cc8ba594`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:28:42.042729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6d05bd9b-d836-48c4-8913-3bddff805df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:42.117340 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1195/2400] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:28:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:28:42.130638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b197f-2595-49cf-9ead-19477f055a17 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:28:42.132610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f48b197f-2595-49cf-9ead-19477f055a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:42.152962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:28:42.226371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b197f-2595-49cf-9ead-19477f055a17 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:28:42.236200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c05eb2-bc28-4ff0-9913-7d0daa261353 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:28:42.236936 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c05eb2-bc28-4ff0-9913-7d0daa261353 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:42.237374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c05eb2-bc28-4ff0-9913-7d0daa261353 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:42.237683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c05eb2-bc28-4ff0-9913-7d0daa261353 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:42.310341 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c05eb2-bc28-4ff0-9913-7d0daa261353 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:28:42.312398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c05eb2-bc28-4ff0-9913-7d0daa261353 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:28:42.324391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1206/2401] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:28:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:28:43.418632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56229cf-fb55-4800-8451-bb32443381fd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:28:43.419823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56229cf-fb55-4800-8451-bb32443381fd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:28:43.420365 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56229cf-fb55-4800-8451-bb32443381fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:28:43.443753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c56229cf-fb55-4800-8451-bb32443381fd None None] MFA Rules not processed for user `fa5bf070765946168f7e5937f2b076c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:28:43.445263 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c56229cf-fb55-4800-8451-bb32443381fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:28:43.459621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1196/2402] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:28:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:28:52.320563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=49 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:28:53.437013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=48 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:29:03.904986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c05eb2-bc28-4ff0-9913-7d0daa261353 service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:03.906506 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-26c05eb2-bc28-4ff0-9913-7d0daa261353 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:03.965231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c05eb2-bc28-4ff0-9913-7d0daa261353 service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:03.968826 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136a11b-35cc-4dce-b84b-c810a4774884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:03.969424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136a11b-35cc-4dce-b84b-c810a4774884 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:03.969700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136a11b-35cc-4dce-b84b-c810a4774884 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:03.969968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136a11b-35cc-4dce-b84b-c810a4774884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:03.971706 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e136a11b-35cc-4dce-b84b-c810a4774884 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:03.985998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136a11b-35cc-4dce-b84b-c810a4774884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:03.987506 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136a11b-35cc-4dce-b84b-c810a4774884 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:03.999293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1207/2403] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:29:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:29:05.183108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56229cf-fb55-4800-8451-bb32443381fd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:05.192747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56229cf-fb55-4800-8451-bb32443381fd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:05.196335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb746c5-238f-4c62-a974-cea031c96497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:05.196920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb746c5-238f-4c62-a974-cea031c96497 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:05.197194 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb746c5-238f-4c62-a974-cea031c96497 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:05.197464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb746c5-238f-4c62-a974-cea031c96497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:05.199194 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8bb746c5-238f-4c62-a974-cea031c96497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:05.212119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb746c5-238f-4c62-a974-cea031c96497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:05.214022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb746c5-238f-4c62-a974-cea031c96497 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:05.224675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1197/2404] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:29:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:29:13.993667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=26 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:29:15.219986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:29:15.873765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136a11b-35cc-4dce-b84b-c810a4774884 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:15.884833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136a11b-35cc-4dce-b84b-c810a4774884 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:15.890312 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39dcd2c-0fe0-4c27-bd30-b4e5e564c3ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:15.890902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dcd2c-0fe0-4c27-bd30-b4e5e564c3ff service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:15.891178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dcd2c-0fe0-4c27-bd30-b4e5e564c3ff service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:15.891449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dcd2c-0fe0-4c27-bd30-b4e5e564c3ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:15.893184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b39dcd2c-0fe0-4c27-bd30-b4e5e564c3ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:15.952698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39dcd2c-0fe0-4c27-bd30-b4e5e564c3ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:15.954325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39dcd2c-0fe0-4c27-bd30-b4e5e564c3ff service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:15.963326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1208/2405] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:29:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:29:18.087195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb746c5-238f-4c62-a974-cea031c96497 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:18.096200 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb746c5-238f-4c62-a974-cea031c96497 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:18.100214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8eb5c6-ae2d-46b3-a9c9-e9121c4f6a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:18.100954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8eb5c6-ae2d-46b3-a9c9-e9121c4f6a3d service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:18.101268 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8eb5c6-ae2d-46b3-a9c9-e9121c4f6a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:18.101567 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8eb5c6-ae2d-46b3-a9c9-e9121c4f6a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:18.104654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9c8eb5c6-ae2d-46b3-a9c9-e9121c4f6a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:18.114900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8eb5c6-ae2d-46b3-a9c9-e9121c4f6a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:18.116327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8eb5c6-ae2d-46b3-a9c9-e9121c4f6a3d service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:18.124828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1198/2406] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:29:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:29:25.959147 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=28 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:29:27.801669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39dcd2c-0fe0-4c27-bd30-b4e5e564c3ff service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:27.810902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39dcd2c-0fe0-4c27-bd30-b4e5e564c3ff service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:27.814506 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab6097-af73-4a42-b5a8-bcafc3be5dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:27.815099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab6097-af73-4a42-b5a8-bcafc3be5dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:27.815375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab6097-af73-4a42-b5a8-bcafc3be5dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:27.815643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab6097-af73-4a42-b5a8-bcafc3be5dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:27.817374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-19ab6097-af73-4a42-b5a8-bcafc3be5dd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:27.826681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab6097-af73-4a42-b5a8-bcafc3be5dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:27.828098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab6097-af73-4a42-b5a8-bcafc3be5dd1 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:27.836828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1209/2407] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:29:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:29:28.121060 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:29:37.833202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=6 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:29:39.536466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa5d53-3c33-431d-afa8-eac45128ceed None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:39.536962 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa5d53-3c33-431d-afa8-eac45128ceed None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:39.538242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa5d53-3c33-431d-afa8-eac45128ceed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:39.566125 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-19fa5d53-3c33-431d-afa8-eac45128ceed None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:29:39.567591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-19fa5d53-3c33-431d-afa8-eac45128ceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:39.623163 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1199/2408] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:29:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:29:39.667774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab6097-af73-4a42-b5a8-bcafc3be5dd1 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:39.669457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-19ab6097-af73-4a42-b5a8-bcafc3be5dd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:39.740801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab6097-af73-4a42-b5a8-bcafc3be5dd1 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:39.746397 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b36474-4a60-4bc3-b701-c4fcabc34db5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:39.747028 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b36474-4a60-4bc3-b701-c4fcabc34db5 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:39.747306 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b36474-4a60-4bc3-b701-c4fcabc34db5 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:39.747575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b36474-4a60-4bc3-b701-c4fcabc34db5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:39.755698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b36474-4a60-4bc3-b701-c4fcabc34db5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:39.757267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b36474-4a60-4bc3-b701-c4fcabc34db5 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:39.765883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1210/2409] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:29:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 27 17:29:46.112752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa5d53-3c33-431d-afa8-eac45128ceed None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:46.130496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa5d53-3c33-431d-afa8-eac45128ceed None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:46.134381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8677c77-ff64-4822-a8eb-f7bcca8f7ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:46.135014 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8677c77-ff64-4822-a8eb-f7bcca8f7ece service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:46.135294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8677c77-ff64-4822-a8eb-f7bcca8f7ece service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:46.135559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8677c77-ff64-4822-a8eb-f7bcca8f7ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:46.145971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8677c77-ff64-4822-a8eb-f7bcca8f7ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:46.147987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8677c77-ff64-4822-a8eb-f7bcca8f7ece service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:46.159302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1200/2410] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:29:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:29:46.875434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b36474-4a60-4bc3-b701-c4fcabc34db5 service swift] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:46.877012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-60b36474-4a60-4bc3-b701-c4fcabc34db5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:46.984109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b36474-4a60-4bc3-b701-c4fcabc34db5 service swift] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:46.991831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497eb86a-0610-4594-8ade-1caf105cfb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:46.992474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eb86a-0610-4594-8ade-1caf105cfb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:46.992746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eb86a-0610-4594-8ade-1caf105cfb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:46.993015 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eb86a-0610-4594-8ade-1caf105cfb7b admin admin] PATH_INFO: `/v3/users/1dba953d778c418abea22efac408e2db` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:46.994212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497eb86a-0610-4594-8ade-1caf105cfb7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dba953d778c418abea22efac408e2db)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:46.997472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497eb86a-0610-4594-8ade-1caf105cfb7b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:47.094871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-497eb86a-0610-4594-8ade-1caf105cfb7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dba953d778c418abea22efac408e2db'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:47.104517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-497eb86a-0610-4594-8ade-1caf105cfb7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dba953d778c418abea22efac408e2db'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:47.105281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-497eb86a-0610-4594-8ade-1caf105cfb7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dba953d778c418abea22efac408e2db'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:47.115507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-497eb86a-0610-4594-8ade-1caf105cfb7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dba953d778c418abea22efac408e2db'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:47.135530 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-497eb86a-0610-4594-8ade-1caf105cfb7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dba953d778c418abea22efac408e2db'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:47.143047 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1211/2411] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:29:46 2022] DELETE /identity/v3/users/1dba953d778c418abea22efac408e2db => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:29:47.474438 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8677c77-ff64-4822-a8eb-f7bcca8f7ece service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:47.480420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8677c77-ff64-4822-a8eb-f7bcca8f7ece service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:47.598001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:47.598839 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:47.599132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:47.599408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608 admin admin] PATH_INFO: `/v3/projects/36351a4bc84a45cb866e72f52e6e725a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:47.601278 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36351a4bc84a45cb866e72f52e6e725a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:47.603858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:47.663979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36351a4bc84a45cb866e72f52e6e725a', 'request_id': 'req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:47.671742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36351a4bc84a45cb866e72f52e6e725a', 'request_id': 'req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:47.680195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36351a4bc84a45cb866e72f52e6e725a', 'request_id': 'req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:47.697665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608 admin admin] The token cache is being invalidate because project 36351a4bc84a45cb866e72f52e6e725a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:29:47.698351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:47.699186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-21dc010a-b53a-4c5b-a9bc-bcccd0eaf608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:47.701144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1201/2412] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:29:47 2022] DELETE /identity/v3/projects/36351a4bc84a45cb866e72f52e6e725a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:29:47.706985 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a68c53-350c-44fa-98a7-987441dbd8db None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:47.707279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a68c53-350c-44fa-98a7-987441dbd8db None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:47.707560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a68c53-350c-44fa-98a7-987441dbd8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:47.732283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d8a68c53-350c-44fa-98a7-987441dbd8db None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:29:47.734706 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d8a68c53-350c-44fa-98a7-987441dbd8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:47.741837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:29:47.822675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1212/2413] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:29:47.830895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c94dcb-838d-4ebf-9095-789c50774d35 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:47.831183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c94dcb-838d-4ebf-9095-789c50774d35 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:47.831463 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c94dcb-838d-4ebf-9095-789c50774d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:47.854351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-35c94dcb-838d-4ebf-9095-789c50774d35 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:29:47.856460 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-35c94dcb-838d-4ebf-9095-789c50774d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:47.866386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1202/2414] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:29:47.871130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a68c53-350c-44fa-98a7-987441dbd8db None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:47.882241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a68c53-350c-44fa-98a7-987441dbd8db None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:47.890650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09ffd5-ae1b-45dd-ac11-4c02ec03652f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:47.891731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09ffd5-ae1b-45dd-ac11-4c02ec03652f admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:47.892170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09ffd5-ae1b-45dd-ac11-4c02ec03652f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:47.892590 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f09ffd5-ae1b-45dd-ac11-4c02ec03652f admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:47.893391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09ffd5-ae1b-45dd-ac11-4c02ec03652f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:29:47.893855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09ffd5-ae1b-45dd-ac11-4c02ec03652f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:47.895624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f09ffd5-ae1b-45dd-ac11-4c02ec03652f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:47.910683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1213/2415] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:29:47.918872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c94dcb-838d-4ebf-9095-789c50774d35 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:47.932013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c94dcb-838d-4ebf-9095-789c50774d35 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:47.936538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b415c-ebea-44bc-a81e-fe3184429784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:47.937188 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b415c-ebea-44bc-a81e-fe3184429784 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:47.937467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b415c-ebea-44bc-a81e-fe3184429784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:47.937738 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b415c-ebea-44bc-a81e-fe3184429784 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:47.938566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b415c-ebea-44bc-a81e-fe3184429784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:47.940736 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b415c-ebea-44bc-a81e-fe3184429784 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:47.960476 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:29:47.983664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1203/2416] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:29:47 2022] POST /identity/v3/projects => generated 399 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:29:47.987590 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09ffd5-ae1b-45dd-ac11-4c02ec03652f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:48.011163 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f09ffd5-ae1b-45dd-ac11-4c02ec03652f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:48.094937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdd3a6-401a-4b10-a6ae-01e4dbd096c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:48.095914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdd3a6-401a-4b10-a6ae-01e4dbd096c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:48.096186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdd3a6-401a-4b10-a6ae-01e4dbd096c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:48.096456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdd3a6-401a-4b10-a6ae-01e4dbd096c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:48.097099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdd3a6-401a-4b10-a6ae-01e4dbd096c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:48.099073 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdd3a6-401a-4b10-a6ae-01e4dbd096c4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:48.124784 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b415c-ebea-44bc-a81e-fe3184429784 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:48.127045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6d8b415c-ebea-44bc-a81e-fe3184429784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:48.139921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1214/2417] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:29:47 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:29:48.144994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdd3a6-401a-4b10-a6ae-01e4dbd096c4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:48.158317 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdd3a6-401a-4b10-a6ae-01e4dbd096c4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:48.167868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e5486-2a28-4cf2-851e-c53e06075345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:48.169760 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e5486-2a28-4cf2-851e-c53e06075345 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:48.170079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e5486-2a28-4cf2-851e-c53e06075345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:48.170414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795e5486-2a28-4cf2-851e-c53e06075345 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:48.172065 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e5486-2a28-4cf2-851e-c53e06075345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:48.176805 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795e5486-2a28-4cf2-851e-c53e06075345 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:48.200830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1215/2418] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:29:48.205426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e5486-2a28-4cf2-851e-c53e06075345 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:48.217407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795e5486-2a28-4cf2-851e-c53e06075345 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:48.224117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763b98b-7e1c-449b-aafc-6e8d6686fdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:48.225496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763b98b-7e1c-449b-aafc-6e8d6686fdce admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:48.225776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763b98b-7e1c-449b-aafc-6e8d6686fdce admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:48.226041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763b98b-7e1c-449b-aafc-6e8d6686fdce admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:48.226706 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763b98b-7e1c-449b-aafc-6e8d6686fdce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:48.229588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763b98b-7e1c-449b-aafc-6e8d6686fdce admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:48.255866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1216/2419] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:29:48.257573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763b98b-7e1c-449b-aafc-6e8d6686fdce admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:48.283072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b415c-ebea-44bc-a81e-fe3184429784 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:48.283640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763b98b-7e1c-449b-aafc-6e8d6686fdce admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:48.292932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4181f1-3cdf-4c25-8499-5dddb0757931 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:48.293666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4181f1-3cdf-4c25-8499-5dddb0757931 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:48.294013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4181f1-3cdf-4c25-8499-5dddb0757931 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:48.294375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4181f1-3cdf-4c25-8499-5dddb0757931 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:48.294714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90518774-80d5-41e9-a012-fef76582a2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:48.295956 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90518774-80d5-41e9-a012-fef76582a2f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:48.296360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5a4181f1-3cdf-4c25-8499-5dddb0757931 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:48.296501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90518774-80d5-41e9-a012-fef76582a2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:48.298152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90518774-80d5-41e9-a012-fef76582a2f1 admin admin] PATH_INFO: `/v3/projects/e8452ed32c514eddae610d28d4aae5e0/users/62e5193078904a5580597ed6ec2b997c/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:48.317568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4181f1-3cdf-4c25-8499-5dddb0757931 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:48.317568 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4181f1-3cdf-4c25-8499-5dddb0757931 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:48.334460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90518774-80d5-41e9-a012-fef76582a2f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8452ed32c514eddae610d28d4aae5e0, user_id=62e5193078904a5580597ed6ec2b997c, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:48.337424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90518774-80d5-41e9-a012-fef76582a2f1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:48.341848 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1204/2420] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:29:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:29:48.359285 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1217/2421] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:29:48 2022] PUT /identity/v3/projects/e8452ed32c514eddae610d28d4aae5e0/users/62e5193078904a5580597ed6ec2b997c/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:29:48.439903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706c74e-9669-4331-8d54-997ba88c7d28 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:48.440460 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706c74e-9669-4331-8d54-997ba88c7d28 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:48.440896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706c74e-9669-4331-8d54-997ba88c7d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:48.496367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9706c74e-9669-4331-8d54-997ba88c7d28 None None] MFA Rules not processed for user `62e5193078904a5580597ed6ec2b997c`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:29:48.499698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9706c74e-9669-4331-8d54-997ba88c7d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:48.610290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1205/2422] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:29:48 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:29:48.623017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90518774-80d5-41e9-a012-fef76582a2f1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:48.624804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-90518774-80d5-41e9-a012-fef76582a2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:48.702879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:29:48.724371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90518774-80d5-41e9-a012-fef76582a2f1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:48.728759 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b452be0-0189-4e51-9d35-11cb2b76c49f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:48.729442 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b452be0-0189-4e51-9d35-11cb2b76c49f service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:48.729722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b452be0-0189-4e51-9d35-11cb2b76c49f service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:48.729997 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b452be0-0189-4e51-9d35-11cb2b76c49f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:48.740653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b452be0-0189-4e51-9d35-11cb2b76c49f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:48.742491 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b452be0-0189-4e51-9d35-11cb2b76c49f service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:48.759019 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1218/2423] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:29:48 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:29:58.156687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706c74e-9669-4331-8d54-997ba88c7d28 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:58.217361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706c74e-9669-4331-8d54-997ba88c7d28 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:58.221870 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:58.222503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:58.222779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:58.223044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] PATH_INFO: `/v3/users/62e5193078904a5580597ed6ec2b997c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:58.224144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e5193078904a5580597ed6ec2b997c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:58.226055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:58.236932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:29:58.272515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e5193078904a5580597ed6ec2b997c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:58.273167 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e5193078904a5580597ed6ec2b997c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:58.280420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e5193078904a5580597ed6ec2b997c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:58.287442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e5193078904a5580597ed6ec2b997c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:58.299802 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e5193078904a5580597ed6ec2b997c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:58.305986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1206/2424] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:29:58 2022] DELETE /identity/v3/users/62e5193078904a5580597ed6ec2b997c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:29:58.310422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49aaea-4ab7-4a45-b692-95ef2fc8535c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:58.310946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49aaea-4ab7-4a45-b692-95ef2fc8535c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:58.310946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49aaea-4ab7-4a45-b692-95ef2fc8535c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:58.329608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5c49aaea-4ab7-4a45-b692-95ef2fc8535c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:29:58.330834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5c49aaea-4ab7-4a45-b692-95ef2fc8535c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:58.389108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1219/2425] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:29:58.398665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:58.400673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:58.456729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45efabc-1605-4ccb-a372-e962439f116b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:58.460429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11347905-cd1c-4b28-8b45-5208742ed7e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:58.461022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11347905-cd1c-4b28-8b45-5208742ed7e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:58.461296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11347905-cd1c-4b28-8b45-5208742ed7e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:58.462187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11347905-cd1c-4b28-8b45-5208742ed7e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:58.471299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11347905-cd1c-4b28-8b45-5208742ed7e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:58.472710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11347905-cd1c-4b28-8b45-5208742ed7e9 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:58.482858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1207/2426] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:29:58.826189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49aaea-4ab7-4a45-b692-95ef2fc8535c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:58.827638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5c49aaea-4ab7-4a45-b692-95ef2fc8535c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:58.853066 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49aaea-4ab7-4a45-b692-95ef2fc8535c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:58.858444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18caff-3a19-4e32-b70b-ee78211dde00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:58.859146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18caff-3a19-4e32-b70b-ee78211dde00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:58.859441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18caff-3a19-4e32-b70b-ee78211dde00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:58.859713 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18caff-3a19-4e32-b70b-ee78211dde00 admin admin] PATH_INFO: `/v3/projects/e8452ed32c514eddae610d28d4aae5e0` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:58.860847 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18caff-3a19-4e32-b70b-ee78211dde00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8452ed32c514eddae610d28d4aae5e0)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:58.862870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18caff-3a19-4e32-b70b-ee78211dde00 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:58.910948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7c18caff-3a19-4e32-b70b-ee78211dde00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8452ed32c514eddae610d28d4aae5e0', 'request_id': 'req-7c18caff-3a19-4e32-b70b-ee78211dde00'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:58.917463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7c18caff-3a19-4e32-b70b-ee78211dde00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8452ed32c514eddae610d28d4aae5e0', 'request_id': 'req-7c18caff-3a19-4e32-b70b-ee78211dde00'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:58.924080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7c18caff-3a19-4e32-b70b-ee78211dde00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8452ed32c514eddae610d28d4aae5e0', 'request_id': 'req-7c18caff-3a19-4e32-b70b-ee78211dde00'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:58.938516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7c18caff-3a19-4e32-b70b-ee78211dde00 admin admin] The token cache is being invalidate because project e8452ed32c514eddae610d28d4aae5e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:29:58.938770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7c18caff-3a19-4e32-b70b-ee78211dde00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:58.939429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7c18caff-3a19-4e32-b70b-ee78211dde00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:29:58.941213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1220/2427] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:29:58 2022] DELETE /identity/v3/projects/e8452ed32c514eddae610d28d4aae5e0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:29:58.947998 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7e036-f213-4958-a8bc-19e22faeda4f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:58.948273 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7e036-f213-4958-a8bc-19e22faeda4f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:58.948538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7e036-f213-4958-a8bc-19e22faeda4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:58.967881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-83f7e036-f213-4958-a8bc-19e22faeda4f None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:29:58.969061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-83f7e036-f213-4958-a8bc-19e22faeda4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:59.029594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1208/2428] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:29:59.037559 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb1e3b-5692-4a1b-aa43-a3ceac040cdf None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:59.037845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb1e3b-5692-4a1b-aa43-a3ceac040cdf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:59.038149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb1e3b-5692-4a1b-aa43-a3ceac040cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:59.060443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8abb1e3b-5692-4a1b-aa43-a3ceac040cdf None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:29:59.061915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8abb1e3b-5692-4a1b-aa43-a3ceac040cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:59.071242 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1221/2429] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:29:59.076387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7e036-f213-4958-a8bc-19e22faeda4f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:59.085841 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7e036-f213-4958-a8bc-19e22faeda4f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:59.090947 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1053cb-1fd6-4653-9786-e5a910bcdfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:59.091533 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1053cb-1fd6-4653-9786-e5a910bcdfde admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:59.091808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1053cb-1fd6-4653-9786-e5a910bcdfde admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:59.092076 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1053cb-1fd6-4653-9786-e5a910bcdfde admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:59.092716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1053cb-1fd6-4653-9786-e5a910bcdfde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:29:59.093013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1053cb-1fd6-4653-9786-e5a910bcdfde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:59.094614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1053cb-1fd6-4653-9786-e5a910bcdfde admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:59.106478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1209/2430] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:29:59.110995 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb1e3b-5692-4a1b-aa43-a3ceac040cdf None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:59.120815 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb1e3b-5692-4a1b-aa43-a3ceac040cdf None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:59.125521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a131e-f167-449c-a55d-e09f7c75a91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:59.126134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a131e-f167-449c-a55d-e09f7c75a91b admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:59.126411 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a131e-f167-449c-a55d-e09f7c75a91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:59.126703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579a131e-f167-449c-a55d-e09f7c75a91b admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:59.127368 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a131e-f167-449c-a55d-e09f7c75a91b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:59.129341 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579a131e-f167-449c-a55d-e09f7c75a91b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:59.157424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1222/2431] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:29:59 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:29:59.161640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1053cb-1fd6-4653-9786-e5a910bcdfde admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:59.171987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1053cb-1fd6-4653-9786-e5a910bcdfde admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:59.184305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:29:59.228001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918950aa-7fb3-4433-8922-56979b6ca87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:59.228522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918950aa-7fb3-4433-8922-56979b6ca87d admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:59.228794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918950aa-7fb3-4433-8922-56979b6ca87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:59.229059 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918950aa-7fb3-4433-8922-56979b6ca87d admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:59.229668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918950aa-7fb3-4433-8922-56979b6ca87d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:59.231858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918950aa-7fb3-4433-8922-56979b6ca87d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:59.252760 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1210/2432] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:29:59 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:29:59.256969 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a131e-f167-449c-a55d-e09f7c75a91b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:59.267794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579a131e-f167-449c-a55d-e09f7c75a91b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:59.272534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7b913-2faa-4441-b30a-f9c8adddb939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:59.273123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7b913-2faa-4441-b30a-f9c8adddb939 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:59.273393 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7b913-2faa-4441-b30a-f9c8adddb939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:59.273654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7b913-2faa-4441-b30a-f9c8adddb939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:59.274353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7b913-2faa-4441-b30a-f9c8adddb939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:59.275954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7b913-2faa-4441-b30a-f9c8adddb939 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:59.286419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1223/2433] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:29:59.290171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918950aa-7fb3-4433-8922-56979b6ca87d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:59.299822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918950aa-7fb3-4433-8922-56979b6ca87d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:59.304416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fc60a-5349-4a7d-8338-aa0f1cd2d888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:59.305073 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fc60a-5349-4a7d-8338-aa0f1cd2d888 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:59.305895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fc60a-5349-4a7d-8338-aa0f1cd2d888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:59.306288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fc60a-5349-4a7d-8338-aa0f1cd2d888 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:59.306932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fc60a-5349-4a7d-8338-aa0f1cd2d888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:59.308505 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fc60a-5349-4a7d-8338-aa0f1cd2d888 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:59.319214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1211/2434] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:29:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:29:59.328110 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7b913-2faa-4441-b30a-f9c8adddb939 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:59.338783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7b913-2faa-4441-b30a-f9c8adddb939 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:59.343492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88099694-6491-4232-a1a9-655f3182817a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:59.344135 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88099694-6491-4232-a1a9-655f3182817a admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:59.344414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88099694-6491-4232-a1a9-655f3182817a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:59.344713 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88099694-6491-4232-a1a9-655f3182817a admin admin] PATH_INFO: `/v3/projects/f4db1a85e2504cb9956a6461b5072aa3/users/cac1528c35fb464db00b5d18396b8eac/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:59.361245 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88099694-6491-4232-a1a9-655f3182817a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4db1a85e2504cb9956a6461b5072aa3, user_id=cac1528c35fb464db00b5d18396b8eac, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:59.364008 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88099694-6491-4232-a1a9-655f3182817a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:59.377226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1224/2435] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:29:59 2022] PUT /identity/v3/projects/f4db1a85e2504cb9956a6461b5072aa3/users/cac1528c35fb464db00b5d18396b8eac/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:29:59.382286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbab61-accd-4016-97cd-419305151028 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:59.382617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbab61-accd-4016-97cd-419305151028 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:59.382894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbbab61-accd-4016-97cd-419305151028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:59.404304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8cbbab61-accd-4016-97cd-419305151028 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:29:59.405509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8cbbab61-accd-4016-97cd-419305151028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:59.472843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1212/2436] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:29:59.483540 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88099694-6491-4232-a1a9-655f3182817a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:29:59.485261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-88099694-6491-4232-a1a9-655f3182817a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:29:59.491026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:29:59.544016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88099694-6491-4232-a1a9-655f3182817a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:29:59.547611 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d30b9-5ab7-4304-9048-52f2bfc6be62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:29:59.548201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d30b9-5ab7-4304-9048-52f2bfc6be62 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:29:59.548472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d30b9-5ab7-4304-9048-52f2bfc6be62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:29:59.548735 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d30b9-5ab7-4304-9048-52f2bfc6be62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:29:59.559399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d30b9-5ab7-4304-9048-52f2bfc6be62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:29:59.560807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d30b9-5ab7-4304-9048-52f2bfc6be62 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:29:59.570019 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1225/2437] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:29:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:30:00.670178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50db326-0eb9-49e6-9aa6-910e192b9fd4 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:00.670770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50db326-0eb9-49e6-9aa6-910e192b9fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:00.671122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50db326-0eb9-49e6-9aa6-910e192b9fd4 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:00.673619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1213/2438] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:30:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:30:00.682204 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad0cb4-9ff4-46a8-b868-5b90a339b741 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:00.683115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad0cb4-9ff4-46a8-b868-5b90a339b741 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:00.683538 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad0cb4-9ff4-46a8-b868-5b90a339b741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:00.707711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-edad0cb4-9ff4-46a8-b868-5b90a339b741 None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:30:00.709418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-edad0cb4-9ff4-46a8-b868-5b90a339b741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:00.798535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1226/2439] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:30:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:30:00.816561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50db326-0eb9-49e6-9aa6-910e192b9fd4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:00.818353 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c50db326-0eb9-49e6-9aa6-910e192b9fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:00.893269 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50db326-0eb9-49e6-9aa6-910e192b9fd4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:00.897264 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffca14f-8fae-4ad5-bfcd-50b62a944661 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc89c1a7e684612a73ec9fef6bda780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:00.898429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffca14f-8fae-4ad5-bfcd-50b62a944661 service glance] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:00.898707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffca14f-8fae-4ad5-bfcd-50b62a944661 service glance] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:00.899610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffca14f-8fae-4ad5-bfcd-50b62a944661 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:00.926671 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffca14f-8fae-4ad5-bfcd-50b62a944661 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:00.926671 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffca14f-8fae-4ad5-bfcd-50b62a944661 service glance] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:00.934470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1214/2440] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:30:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:30:01.043496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f0607-f1f8-4f92-9c7c-56464539eb68 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:01.043923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f0607-f1f8-4f92-9c7c-56464539eb68 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:01.044297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f0607-f1f8-4f92-9c7c-56464539eb68 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:01.046117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1227/2441] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:30:01.051429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166dbf9-d57e-4f39-98a6-dd43c7e358a1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:01.051721 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166dbf9-d57e-4f39-98a6-dd43c7e358a1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:01.052326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166dbf9-d57e-4f39-98a6-dd43c7e358a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:01.083687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2166dbf9-d57e-4f39-98a6-dd43c7e358a1 None None] MFA Rules not processed for user `5d906452199844149966c9c86db4684e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:30:01.084860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2166dbf9-d57e-4f39-98a6-dd43c7e358a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:01.107910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1215/2442] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:30:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:30:01.108340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f0607-f1f8-4f92-9c7c-56464539eb68 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:01.110246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6b5f0607-f1f8-4f92-9c7c-56464539eb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:01.125754 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f0607-f1f8-4f92-9c7c-56464539eb68 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:01.129559 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec97025-9324-464c-9269-4f43ceb7c59a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:01.130271 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec97025-9324-464c-9269-4f43ceb7c59a service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:01.130621 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec97025-9324-464c-9269-4f43ceb7c59a service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:01.130968 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec97025-9324-464c-9269-4f43ceb7c59a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:01.142183 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec97025-9324-464c-9269-4f43ceb7c59a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:01.143627 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec97025-9324-464c-9269-4f43ceb7c59a service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:01.153384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1228/2443] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:30:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:30:01.283095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55eff62-693a-49f6-83a5-43f453479979 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:01.283492 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55eff62-693a-49f6-83a5-43f453479979 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:01.283839 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55eff62-693a-49f6-83a5-43f453479979 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:01.285185 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1216/2444] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:30:01.290357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c4ff7-9b57-4094-96a9-4f488eaea2bb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:01.290861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c4ff7-9b57-4094-96a9-4f488eaea2bb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:01.290986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c4ff7-9b57-4094-96a9-4f488eaea2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:01.325641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f36c4ff7-9b57-4094-96a9-4f488eaea2bb None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:30:01.327612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f36c4ff7-9b57-4094-96a9-4f488eaea2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:01.341208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1229/2445] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:30:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:30:01.352630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55eff62-693a-49f6-83a5-43f453479979 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:01.367050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55eff62-693a-49f6-83a5-43f453479979 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:01.374245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b3f201-0d1e-4e8b-856d-9871ec96ed6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:01.374966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3f201-0d1e-4e8b-856d-9871ec96ed6c service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:01.375938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3f201-0d1e-4e8b-856d-9871ec96ed6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:01.376210 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b3f201-0d1e-4e8b-856d-9871ec96ed6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:01.391496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3f201-0d1e-4e8b-856d-9871ec96ed6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:01.395756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b3f201-0d1e-4e8b-856d-9871ec96ed6c service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:01.412144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1217/2446] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:30:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:30:01.487468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c4ff7-9b57-4094-96a9-4f488eaea2bb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:01.490586 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f36c4ff7-9b57-4094-96a9-4f488eaea2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:01.569729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c4ff7-9b57-4094-96a9-4f488eaea2bb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:01.580182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721ea5e-efc0-4d37-8f6d-44553d841323 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:01.580765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721ea5e-efc0-4d37-8f6d-44553d841323 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:01.581113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721ea5e-efc0-4d37-8f6d-44553d841323 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:01.581451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721ea5e-efc0-4d37-8f6d-44553d841323 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:01.583486 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b721ea5e-efc0-4d37-8f6d-44553d841323 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:01.596787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b721ea5e-efc0-4d37-8f6d-44553d841323 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:01.598759 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b721ea5e-efc0-4d37-8f6d-44553d841323 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:01.609832 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1230/2447] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:30:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:30:03.927378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d06ab-ec02-43ab-9c94-a912052f0037 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:03.927857 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d06ab-ec02-43ab-9c94-a912052f0037 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:03.928673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d06ab-ec02-43ab-9c94-a912052f0037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:03.959633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f08d06ab-ec02-43ab-9c94-a912052f0037 None None] MFA Rules not processed for user `cac1528c35fb464db00b5d18396b8eac`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:30:03.961571 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f08d06ab-ec02-43ab-9c94-a912052f0037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:03.972137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:30:04.040413 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1218/2448] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:30:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:30:04.050302 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721ea5e-efc0-4d37-8f6d-44553d841323 service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:04.069391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721ea5e-efc0-4d37-8f6d-44553d841323 service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:04.077992 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7b1af-54f7-4bca-adbd-a212c9c6d2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:04.078720 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7b1af-54f7-4bca-adbd-a212c9c6d2ac admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:04.079063 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7b1af-54f7-4bca-adbd-a212c9c6d2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:04.079425 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7b1af-54f7-4bca-adbd-a212c9c6d2ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:04.080170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7b1af-54f7-4bca-adbd-a212c9c6d2ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:04.082062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7b1af-54f7-4bca-adbd-a212c9c6d2ac admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:04.086867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:30:04.124305 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1231/2449] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:30:04 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:30:04.128581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d06ab-ec02-43ab-9c94-a912052f0037 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:04.140228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d06ab-ec02-43ab-9c94-a912052f0037 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:04.205706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b3e74-1056-4553-ad72-4d466c56e297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:04.206361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b3e74-1056-4553-ad72-4d466c56e297 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:04.206638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b3e74-1056-4553-ad72-4d466c56e297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:04.206908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7b3e74-1056-4553-ad72-4d466c56e297 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:04.207563 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b3e74-1056-4553-ad72-4d466c56e297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:04.209509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7b3e74-1056-4553-ad72-4d466c56e297 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:04.235197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1219/2450] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:30:04 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:30:04.239456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7b1af-54f7-4bca-adbd-a212c9c6d2ac admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:04.253070 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7b1af-54f7-4bca-adbd-a212c9c6d2ac admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:04.258907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a55d9-c4d9-483d-9236-059f10114794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:04.259691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a55d9-c4d9-483d-9236-059f10114794 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:04.281201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a55d9-c4d9-483d-9236-059f10114794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:04.281607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a55d9-c4d9-483d-9236-059f10114794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:04.282557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a55d9-c4d9-483d-9236-059f10114794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:04.284809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a55d9-c4d9-483d-9236-059f10114794 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:04.305863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1232/2451] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:30:04.309418 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b3e74-1056-4553-ad72-4d466c56e297 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:04.322995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7b3e74-1056-4553-ad72-4d466c56e297 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:04.334908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba99953-5c8d-4282-af7e-480282add000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:04.335598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba99953-5c8d-4282-af7e-480282add000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:04.335873 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba99953-5c8d-4282-af7e-480282add000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:04.336162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba99953-5c8d-4282-af7e-480282add000 admin admin] PATH_INFO: `/v3/projects/c26e4ed5e0bc4eb5ad8b3f79e56b2424/users/6f41a64b57de459c8bb764bf7eb3737c/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:04.355818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba99953-5c8d-4282-af7e-480282add000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26e4ed5e0bc4eb5ad8b3f79e56b2424, user_id=6f41a64b57de459c8bb764bf7eb3737c, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:04.358923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba99953-5c8d-4282-af7e-480282add000 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:04.375326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1220/2452] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:30:04 2022] PUT /identity/v3/projects/c26e4ed5e0bc4eb5ad8b3f79e56b2424/users/6f41a64b57de459c8bb764bf7eb3737c/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:30:04.379278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a55d9-c4d9-483d-9236-059f10114794 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:04.391458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a55d9-c4d9-483d-9236-059f10114794 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:04.459796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071da610-9104-4419-95e8-6cd6ebea2ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:04.460292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071da610-9104-4419-95e8-6cd6ebea2ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:04.460568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071da610-9104-4419-95e8-6cd6ebea2ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:04.460834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071da610-9104-4419-95e8-6cd6ebea2ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:04.462373 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071da610-9104-4419-95e8-6cd6ebea2ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:04.464711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071da610-9104-4419-95e8-6cd6ebea2ffa admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:04.475738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1233/2453] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:30:04.480118 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba99953-5c8d-4282-af7e-480282add000 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:04.495251 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba99953-5c8d-4282-af7e-480282add000 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:04.502822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a03fdc-0fc9-4d08-87e2-ee748b19f3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:04.503601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a03fdc-0fc9-4d08-87e2-ee748b19f3b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:04.503942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a03fdc-0fc9-4d08-87e2-ee748b19f3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:04.504279 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a03fdc-0fc9-4d08-87e2-ee748b19f3b1 admin admin] PATH_INFO: `/v3/projects/c26e4ed5e0bc4eb5ad8b3f79e56b2424/users/6f41a64b57de459c8bb764bf7eb3737c/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:04.507008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a03fdc-0fc9-4d08-87e2-ee748b19f3b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26e4ed5e0bc4eb5ad8b3f79e56b2424, user_id=6f41a64b57de459c8bb764bf7eb3737c, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:04.510637 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a03fdc-0fc9-4d08-87e2-ee748b19f3b1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:04.530562 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1221/2454] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:30:04 2022] PUT /identity/v3/projects/c26e4ed5e0bc4eb5ad8b3f79e56b2424/users/6f41a64b57de459c8bb764bf7eb3737c/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:30:05.048943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d176a-45e1-4175-bfd9-4b4c784edfb4 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:05.049229 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d176a-45e1-4175-bfd9-4b4c784edfb4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:05.049501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d176a-45e1-4175-bfd9-4b4c784edfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:05.070997 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b29d176a-45e1-4175-bfd9-4b4c784edfb4 None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:30:05.072140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b29d176a-45e1-4175-bfd9-4b4c784edfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:05.145459 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1234/2455] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:30:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:30:05.160630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a03fdc-0fc9-4d08-87e2-ee748b19f3b1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:05.162588 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-50a03fdc-0fc9-4d08-87e2-ee748b19f3b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:05.263856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:30:05.269828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a03fdc-0fc9-4d08-87e2-ee748b19f3b1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:05.283289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a47c3-601f-4f11-8225-31e050848c49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:05.283931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a47c3-601f-4f11-8225-31e050848c49 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:05.284243 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a47c3-601f-4f11-8225-31e050848c49 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:05.284570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a47c3-601f-4f11-8225-31e050848c49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:05.298828 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a47c3-601f-4f11-8225-31e050848c49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:05.301054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a47c3-601f-4f11-8225-31e050848c49 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:05.315956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1222/2456] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:30:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:30:11.325515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a638ad-6ad0-4e76-930f-ae4aec42d371 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:11.326219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a638ad-6ad0-4e76-930f-ae4aec42d371 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:11.326219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a638ad-6ad0-4e76-930f-ae4aec42d371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:11.373212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a0a638ad-6ad0-4e76-930f-ae4aec42d371 None None] MFA Rules not processed for user `6f41a64b57de459c8bb764bf7eb3737c`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:30:11.374470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a0a638ad-6ad0-4e76-930f-ae4aec42d371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:11.461051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:30:11.474355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1235/2457] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:30:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:30:11.484096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a47c3-601f-4f11-8225-31e050848c49 service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:11.485588 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2d6a47c3-601f-4f11-8225-31e050848c49 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:11.662963 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a47c3-601f-4f11-8225-31e050848c49 service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:11.669794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664af0a-6e2b-4f35-8e78-33863b712324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:11.670851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664af0a-6e2b-4f35-8e78-33863b712324 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:11.671370 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664af0a-6e2b-4f35-8e78-33863b712324 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:11.671911 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664af0a-6e2b-4f35-8e78-33863b712324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:11.792345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664af0a-6e2b-4f35-8e78-33863b712324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:11.794178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664af0a-6e2b-4f35-8e78-33863b712324 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:11.821022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1223/2458] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:30:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:30:21.816772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=60 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:30:27.151415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfea90-dc07-47dd-b6a2-f899f78c6d55 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:27.151990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfea90-dc07-47dd-b6a2-f899f78c6d55 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:27.151990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfea90-dc07-47dd-b6a2-f899f78c6d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:27.171286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7abfea90-dc07-47dd-b6a2-f899f78c6d55 None None] MFA Rules not processed for user `0b02c2ba9a0a48c294a11b42cbfe28e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:30:27.172480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7abfea90-dc07-47dd-b6a2-f899f78c6d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:27.225607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1236/2459] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:30:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:30:27.536024 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664af0a-6e2b-4f35-8e78-33863b712324 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:27.589457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664af0a-6e2b-4f35-8e78-33863b712324 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:27.593145 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e7e28-9c0d-4346-bd43-b9c509f050fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:27.593736 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e7e28-9c0d-4346-bd43-b9c509f050fd service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:27.594008 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e7e28-9c0d-4346-bd43-b9c509f050fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:27.594297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e7e28-9c0d-4346-bd43-b9c509f050fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:27.595979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-123e7e28-9c0d-4346-bd43-b9c509f050fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:27.605301 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e7e28-9c0d-4346-bd43-b9c509f050fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:27.606977 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e7e28-9c0d-4346-bd43-b9c509f050fd service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:27.614708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1224/2460] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:30:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:30:32.929799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfea90-dc07-47dd-b6a2-f899f78c6d55 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:32.932677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7abfea90-dc07-47dd-b6a2-f899f78c6d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:32.964848 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfea90-dc07-47dd-b6a2-f899f78c6d55 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:32.973226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b56938-d057-4927-9390-b858a83cabfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:32.974371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b56938-d057-4927-9390-b858a83cabfa service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:32.975277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b56938-d057-4927-9390-b858a83cabfa service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:32.975769 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b56938-d057-4927-9390-b858a83cabfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:32.979772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-23b56938-d057-4927-9390-b858a83cabfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:33.005434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b56938-d057-4927-9390-b858a83cabfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:33.007627 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b56938-d057-4927-9390-b858a83cabfa service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:33.022132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1237/2461] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:30:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:30:34.848780 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e7e28-9c0d-4346-bd43-b9c509f050fd service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:34.864024 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e7e28-9c0d-4346-bd43-b9c509f050fd service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:34.876612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cd92c-77eb-4994-8003-1c934a6d339c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:34.877327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cd92c-77eb-4994-8003-1c934a6d339c service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:34.877670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cd92c-77eb-4994-8003-1c934a6d339c service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:34.878004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cd92c-77eb-4994-8003-1c934a6d339c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:34.880875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-014cd92c-77eb-4994-8003-1c934a6d339c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:34.896985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cd92c-77eb-4994-8003-1c934a6d339c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:34.900337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cd92c-77eb-4994-8003-1c934a6d339c service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:34.912778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1225/2462] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:30:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:30:42.031156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c41257-9f2e-4baa-84d8-182ef7a9adef None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:42.031628 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c41257-9f2e-4baa-84d8-182ef7a9adef None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:42.031744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c41257-9f2e-4baa-84d8-182ef7a9adef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:42.050990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e1c41257-9f2e-4baa-84d8-182ef7a9adef None None] MFA Rules not processed for user `cac1528c35fb464db00b5d18396b8eac`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:30:42.052180 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e1c41257-9f2e-4baa-84d8-182ef7a9adef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:42.059593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1238/2463] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:30:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:30:42.453193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2733d4-da88-4346-b08e-d9a8078822c4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:42.453775 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2733d4-da88-4346-b08e-d9a8078822c4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:42.453860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2733d4-da88-4346-b08e-d9a8078822c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:42.482918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8d2733d4-da88-4346-b08e-d9a8078822c4 None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:30:42.484767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8d2733d4-da88-4346-b08e-d9a8078822c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:42.492488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1226/2464] 10.222.0.55 () {58 vars in 967 bytes} [Tue Sep 27 17:30:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:30:42.516178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c41257-9f2e-4baa-84d8-182ef7a9adef None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:42.528118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c41257-9f2e-4baa-84d8-182ef7a9adef None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:42.534480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a75e5-fd11-4a14-a6ac-fc562d945c5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:42.535090 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a75e5-fd11-4a14-a6ac-fc562d945c5e service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:42.535373 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a75e5-fd11-4a14-a6ac-fc562d945c5e service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:42.535666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a75e5-fd11-4a14-a6ac-fc562d945c5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:42.544655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a75e5-fd11-4a14-a6ac-fc562d945c5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:42.546201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a75e5-fd11-4a14-a6ac-fc562d945c5e service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:42.554273 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1239/2465] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:30:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:30:48.262252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2733d4-da88-4346-b08e-d9a8078822c4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:30:48.278940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2733d4-da88-4346-b08e-d9a8078822c4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:30:48.285424 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722e5af-dd26-49ae-9ff6-1452a1ca6ed3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:30:48.286272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e5af-dd26-49ae-9ff6-1452a1ca6ed3 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:30:48.286831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e5af-dd26-49ae-9ff6-1452a1ca6ed3 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:30:48.287214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e5af-dd26-49ae-9ff6-1452a1ca6ed3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:30:48.289285 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3722e5af-dd26-49ae-9ff6-1452a1ca6ed3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:30:48.354197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722e5af-dd26-49ae-9ff6-1452a1ca6ed3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:30:48.355844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722e5af-dd26-49ae-9ff6-1452a1ca6ed3 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:30:48.365317 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1227/2466] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:30:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:30:52.551005 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=40 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:30:58.361034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=63 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:31:05.427382 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a75e5-fd11-4a14-a6ac-fc562d945c5e service swift] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:31:05.436669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a75e5-fd11-4a14-a6ac-fc562d945c5e service swift] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:31:05.440350 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1a686-aaf2-4ad2-a78a-d5eabb073a01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:31:05.440943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1a686-aaf2-4ad2-a78a-d5eabb073a01 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:31:05.441220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1a686-aaf2-4ad2-a78a-d5eabb073a01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:31:05.441491 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1a686-aaf2-4ad2-a78a-d5eabb073a01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:31:05.450525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1a686-aaf2-4ad2-a78a-d5eabb073a01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:31:05.451951 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f1a686-aaf2-4ad2-a78a-d5eabb073a01 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:31:05.462210 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1240/2467] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:31:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:31:15.457146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=6 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:31:22.011525 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35cb1e-6f53-4cab-add5-02c4c6018da1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:31:22.012055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35cb1e-6f53-4cab-add5-02c4c6018da1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:31:22.012055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35cb1e-6f53-4cab-add5-02c4c6018da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:31:22.032872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ed35cb1e-6f53-4cab-add5-02c4c6018da1 None None] MFA Rules not processed for user `d23a498bde00483c8fdd68dcde975e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:31:22.034064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ed35cb1e-6f53-4cab-add5-02c4c6018da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:31:22.041559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1228/2468] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:31:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:31:32.026258 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:32:20.847523 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1a686-aaf2-4ad2-a78a-d5eabb073a01 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:32:21.001786 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1a686-aaf2-4ad2-a78a-d5eabb073a01 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:32:21.006826 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b76288-9231-43e4-971f-edf6799769ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:32:21.007434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b76288-9231-43e4-971f-edf6799769ee service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:32:21.007760 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b76288-9231-43e4-971f-edf6799769ee service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:32:21.007973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b76288-9231-43e4-971f-edf6799769ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:32:21.009949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-78b76288-9231-43e4-971f-edf6799769ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:32:21.046495 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b76288-9231-43e4-971f-edf6799769ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:32:21.048096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b76288-9231-43e4-971f-edf6799769ee service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:32:21.059532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1241/2469] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:32:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:32:23.862976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e108f-3e2e-4332-8f8c-128b3f957bf1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:32:23.863479 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e108f-3e2e-4332-8f8c-128b3f957bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:32:23.863672 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e108f-3e2e-4332-8f8c-128b3f957bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:32:23.883122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-958e108f-3e2e-4332-8f8c-128b3f957bf1 None None] MFA Rules not processed for user `1dde550901f04ea695b85db4e41d9628`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:32:23.884342 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-958e108f-3e2e-4332-8f8c-128b3f957bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:32:23.937489 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1229/2470] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:32:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:32:26.213878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b76288-9231-43e4-971f-edf6799769ee service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:32:26.243156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b76288-9231-43e4-971f-edf6799769ee service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:32:26.253816 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c35a5d-f135-45ad-9632-85988e638ef9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:32:26.254487 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c35a5d-f135-45ad-9632-85988e638ef9 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:32:26.254751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c35a5d-f135-45ad-9632-85988e638ef9 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:32:26.255019 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c35a5d-f135-45ad-9632-85988e638ef9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:32:26.256771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a5c35a5d-f135-45ad-9632-85988e638ef9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:32:26.322726 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c35a5d-f135-45ad-9632-85988e638ef9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:32:26.324327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c35a5d-f135-45ad-9632-85988e638ef9 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:32:26.333525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1242/2471] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:32:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:32:33.930842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=25 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:32:36.329239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:32:40.502312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84239743-b2fc-48f5-8cd0-362fb44a6fa1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:32:40.502899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84239743-b2fc-48f5-8cd0-362fb44a6fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:32:40.502899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84239743-b2fc-48f5-8cd0-362fb44a6fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:32:40.538749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-84239743-b2fc-48f5-8cd0-362fb44a6fa1 None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:32:40.539987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-84239743-b2fc-48f5-8cd0-362fb44a6fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:32:40.547830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1230/2472] 10.222.0.55 () {58 vars in 967 bytes} [Tue Sep 27 17:32:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:32:40.604481 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c35a5d-f135-45ad-9632-85988e638ef9 service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:32:40.618047 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c35a5d-f135-45ad-9632-85988e638ef9 service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:32:40.624174 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab113b8a-c4c0-415b-b56f-73c6d2d46cca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:32:40.624978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab113b8a-c4c0-415b-b56f-73c6d2d46cca service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:32:40.625577 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab113b8a-c4c0-415b-b56f-73c6d2d46cca service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:32:40.626000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab113b8a-c4c0-415b-b56f-73c6d2d46cca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:32:40.635472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab113b8a-c4c0-415b-b56f-73c6d2d46cca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:32:40.637340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab113b8a-c4c0-415b-b56f-73c6d2d46cca service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:32:40.648474 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1243/2473] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:32:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:32:41.043977 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84239743-b2fc-48f5-8cd0-362fb44a6fa1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:32:41.064290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84239743-b2fc-48f5-8cd0-362fb44a6fa1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:32:41.068083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f32c4f-4f8e-4750-a4b4-2fc5c8b2d9e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:32:41.068689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f32c4f-4f8e-4750-a4b4-2fc5c8b2d9e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:32:41.068993 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f32c4f-4f8e-4750-a4b4-2fc5c8b2d9e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:32:41.069262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f32c4f-4f8e-4750-a4b4-2fc5c8b2d9e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:32:41.071102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-05f32c4f-4f8e-4750-a4b4-2fc5c8b2d9e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:32:41.178564 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f32c4f-4f8e-4750-a4b4-2fc5c8b2d9e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:32:41.180025 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f32c4f-4f8e-4750-a4b4-2fc5c8b2d9e9 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:32:41.200121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1231/2474] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:32:44.657379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab113b8a-c4c0-415b-b56f-73c6d2d46cca service swift] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:32:44.669966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab113b8a-c4c0-415b-b56f-73c6d2d46cca service swift] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:32:44.675366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf59d10-54a1-498b-a540-b02844beccfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:32:44.675972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf59d10-54a1-498b-a540-b02844beccfd service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:32:44.676243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf59d10-54a1-498b-a540-b02844beccfd service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:32:44.676679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf59d10-54a1-498b-a540-b02844beccfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:32:44.678759 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2cf59d10-54a1-498b-a540-b02844beccfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:32:44.690612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf59d10-54a1-498b-a540-b02844beccfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:32:44.692243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf59d10-54a1-498b-a540-b02844beccfd service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:32:44.703424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1244/2475] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:32:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:32:45.906443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f32c4f-4f8e-4750-a4b4-2fc5c8b2d9e9 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:32:45.927304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f32c4f-4f8e-4750-a4b4-2fc5c8b2d9e9 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:32:45.932840 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be537fb6-beb5-4cd6-a954-b5b583a50abc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:32:45.933445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be537fb6-beb5-4cd6-a954-b5b583a50abc service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:32:45.933757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be537fb6-beb5-4cd6-a954-b5b583a50abc service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:32:45.934033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be537fb6-beb5-4cd6-a954-b5b583a50abc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:32:45.936378 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-be537fb6-beb5-4cd6-a954-b5b583a50abc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:32:45.947051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be537fb6-beb5-4cd6-a954-b5b583a50abc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:32:45.948626 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be537fb6-beb5-4cd6-a954-b5b583a50abc service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:32:45.958442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1232/2476] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:32:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:32:48.153304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf59d10-54a1-498b-a540-b02844beccfd service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:32:48.164375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf59d10-54a1-498b-a540-b02844beccfd service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:32:48.169321 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1b5f1-cd4c-4638-a9e1-946dac64de93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:32:48.170599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1b5f1-cd4c-4638-a9e1-946dac64de93 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:32:48.170875 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1b5f1-cd4c-4638-a9e1-946dac64de93 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:32:48.171141 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea1b5f1-cd4c-4638-a9e1-946dac64de93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:32:48.172873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cea1b5f1-cd4c-4638-a9e1-946dac64de93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:32:48.184594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea1b5f1-cd4c-4638-a9e1-946dac64de93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:32:48.186442 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea1b5f1-cd4c-4638-a9e1-946dac64de93 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:32:48.196925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1245/2477] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:32:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:32:48.277030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be537fb6-beb5-4cd6-a954-b5b583a50abc service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:32:48.278538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-be537fb6-beb5-4cd6-a954-b5b583a50abc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:32:48.292906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be537fb6-beb5-4cd6-a954-b5b583a50abc service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:32:48.297415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:32:48.298035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:32:48.298343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:32:48.298609 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] PATH_INFO: `/v3/users/c3acafe7ed414c87a5ca235b96047aca` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:32:48.299751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3acafe7ed414c87a5ca235b96047aca)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:32:48.301695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:32:48.356079 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3acafe7ed414c87a5ca235b96047aca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:48.356993 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3acafe7ed414c87a5ca235b96047aca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:48.364111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3acafe7ed414c87a5ca235b96047aca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:48.371460 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3acafe7ed414c87a5ca235b96047aca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:48.384179 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3acafe7ed414c87a5ca235b96047aca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:48.386641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:32:48.392759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1233/2478] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:32:48 2022] DELETE /identity/v3/users/c3acafe7ed414c87a5ca235b96047aca => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:32:48.591379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1b5f1-cd4c-4638-a9e1-946dac64de93 service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:32:48.604115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea1b5f1-cd4c-4638-a9e1-946dac64de93 service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:32:48.674862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:32:48.675648 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:32:48.675995 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:32:48.676340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] PATH_INFO: `/v3/projects/9bf4e20a59f147afb4f036fc8d04adb2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:32:48.677580 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf4e20a59f147afb4f036fc8d04adb2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:32:48.679738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:32:48.739636 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf4e20a59f147afb4f036fc8d04adb2', 'request_id': 'req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:48.749349 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf4e20a59f147afb4f036fc8d04adb2', 'request_id': 'req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:48.756744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf4e20a59f147afb4f036fc8d04adb2', 'request_id': 'req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:48.783000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] The token cache is being invalidate because project 9bf4e20a59f147afb4f036fc8d04adb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:32:48.783320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:48.783987 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:48.787253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1246/2479] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:32:48 2022] DELETE /identity/v3/projects/9bf4e20a59f147afb4f036fc8d04adb2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:32:48.791296 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:32:48.799839 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:32:48.834145 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c48e93-bd62-44e2-af54-2c6ff9e45d12 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:32:48.931238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a16dd-39ef-4890-9769-d4241039b9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:32:48.932697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a16dd-39ef-4890-9769-d4241039b9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:32:48.932878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a16dd-39ef-4890-9769-d4241039b9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:32:48.933160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a16dd-39ef-4890-9769-d4241039b9aa admin admin] PATH_INFO: `/v3/users/8d0791cc553a41deadf1dd0c0195482b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:32:48.934468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a16dd-39ef-4890-9769-d4241039b9aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d0791cc553a41deadf1dd0c0195482b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:32:48.936880 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a16dd-39ef-4890-9769-d4241039b9aa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:32:49.025935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-079a16dd-39ef-4890-9769-d4241039b9aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d0791cc553a41deadf1dd0c0195482b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:49.026844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-079a16dd-39ef-4890-9769-d4241039b9aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d0791cc553a41deadf1dd0c0195482b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:49.035515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-079a16dd-39ef-4890-9769-d4241039b9aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d0791cc553a41deadf1dd0c0195482b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:49.048966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-079a16dd-39ef-4890-9769-d4241039b9aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d0791cc553a41deadf1dd0c0195482b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:49.075570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-079a16dd-39ef-4890-9769-d4241039b9aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d0791cc553a41deadf1dd0c0195482b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:49.083488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1234/2480] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:32:48 2022] DELETE /identity/v3/users/8d0791cc553a41deadf1dd0c0195482b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:32:49.363626 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:32:49.370404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:32:49.386022 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3a91-a5bc-4f6f-9c4a-f89cec29bb88 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:32:49.443119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:32:49.454732 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:32:49.455466 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:32:49.455808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:32:49.456168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] PATH_INFO: `/v3/projects/693365fee2754e3e8ac69052733795a3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:32:49.457403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=693365fee2754e3e8ac69052733795a3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:32:49.459433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:32:49.511994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '693365fee2754e3e8ac69052733795a3', 'request_id': 'req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:49.519106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '693365fee2754e3e8ac69052733795a3', 'request_id': 'req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:49.526667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '693365fee2754e3e8ac69052733795a3', 'request_id': 'req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:49.540606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] The token cache is being invalidate because project 693365fee2754e3e8ac69052733795a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:32:49.540977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:49.541555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:32:49.543078 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1247/2481] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:32:49 2022] DELETE /identity/v3/projects/693365fee2754e3e8ac69052733795a3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:32:59.079386 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:32:59.537269 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:04.952514 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bc095-55f6-45d8-8116-c53c46fe097c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:04.953072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bc095-55f6-45d8-8116-c53c46fe097c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:04.953072 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bc095-55f6-45d8-8116-c53c46fe097c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:04.973316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e81bc095-55f6-45d8-8116-c53c46fe097c None None] MFA Rules not processed for user `0b02c2ba9a0a48c294a11b42cbfe28e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:04.974582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e81bc095-55f6-45d8-8116-c53c46fe097c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:05.032506 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1235/2482] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:05.664903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:05.667143 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:05.767490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383f45a-44f0-411c-a1f2-cbd23dd8e4e6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:05.771871 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2d773-2f7c-4ac1-900d-8e032490d56c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:05.772477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2d773-2f7c-4ac1-900d-8e032490d56c service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:05.772751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2d773-2f7c-4ac1-900d-8e032490d56c service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:05.773631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2d773-2f7c-4ac1-900d-8e032490d56c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:05.775909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-20c2d773-2f7c-4ac1-900d-8e032490d56c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:05.843951 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2d773-2f7c-4ac1-900d-8e032490d56c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:05.845705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2d773-2f7c-4ac1-900d-8e032490d56c service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:05.857067 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1248/2483] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:09.732176 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bc095-55f6-45d8-8116-c53c46fe097c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:09.733728 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e81bc095-55f6-45d8-8116-c53c46fe097c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:09.755359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bc095-55f6-45d8-8116-c53c46fe097c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:09.766663 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69feb8-af76-47d9-95b7-369bbee0b613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:09.767308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69feb8-af76-47d9-95b7-369bbee0b613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:09.767581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69feb8-af76-47d9-95b7-369bbee0b613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:09.767881 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69feb8-af76-47d9-95b7-369bbee0b613 admin admin] PATH_INFO: `/v3/users/d23a498bde00483c8fdd68dcde975e5d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:09.769057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69feb8-af76-47d9-95b7-369bbee0b613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23a498bde00483c8fdd68dcde975e5d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:09.771029 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69feb8-af76-47d9-95b7-369bbee0b613 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:09.818844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7c69feb8-af76-47d9-95b7-369bbee0b613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23a498bde00483c8fdd68dcde975e5d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:09.819894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7c69feb8-af76-47d9-95b7-369bbee0b613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23a498bde00483c8fdd68dcde975e5d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:09.828514 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7c69feb8-af76-47d9-95b7-369bbee0b613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23a498bde00483c8fdd68dcde975e5d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:09.836425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7c69feb8-af76-47d9-95b7-369bbee0b613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23a498bde00483c8fdd68dcde975e5d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:09.849440 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7c69feb8-af76-47d9-95b7-369bbee0b613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23a498bde00483c8fdd68dcde975e5d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:09.855407 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1236/2484] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:33:09 2022] DELETE /identity/v3/users/d23a498bde00483c8fdd68dcde975e5d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:10.035685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2d773-2f7c-4ac1-900d-8e032490d56c service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:10.051938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2d773-2f7c-4ac1-900d-8e032490d56c service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:10.119429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ffd81-453c-4653-b3b8-8c95d33a3761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:10.120097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ffd81-453c-4653-b3b8-8c95d33a3761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:10.120373 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ffd81-453c-4653-b3b8-8c95d33a3761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:10.120651 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ffd81-453c-4653-b3b8-8c95d33a3761 admin admin] PATH_INFO: `/v3/projects/fd5caa5782e74721a4c2991524182668` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:10.121789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ffd81-453c-4653-b3b8-8c95d33a3761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd5caa5782e74721a4c2991524182668)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:10.123819 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ffd81-453c-4653-b3b8-8c95d33a3761 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:10.166728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:10.175573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4c8ffd81-453c-4653-b3b8-8c95d33a3761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd5caa5782e74721a4c2991524182668', 'request_id': 'req-4c8ffd81-453c-4653-b3b8-8c95d33a3761'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:10.181345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4c8ffd81-453c-4653-b3b8-8c95d33a3761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd5caa5782e74721a4c2991524182668', 'request_id': 'req-4c8ffd81-453c-4653-b3b8-8c95d33a3761'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:10.187775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4c8ffd81-453c-4653-b3b8-8c95d33a3761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd5caa5782e74721a4c2991524182668', 'request_id': 'req-4c8ffd81-453c-4653-b3b8-8c95d33a3761'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:10.201051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4c8ffd81-453c-4653-b3b8-8c95d33a3761 admin admin] The token cache is being invalidate because project fd5caa5782e74721a4c2991524182668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:33:10.201315 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4c8ffd81-453c-4653-b3b8-8c95d33a3761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:10.201934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4c8ffd81-453c-4653-b3b8-8c95d33a3761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:10.203452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1249/2485] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:33:10 2022] DELETE /identity/v3/projects/fd5caa5782e74721a4c2991524182668 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:10.214583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeae4c-a7bd-4b11-8e6f-bbf89d40ea54 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:10.214877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeae4c-a7bd-4b11-8e6f-bbf89d40ea54 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:10.215144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeae4c-a7bd-4b11-8e6f-bbf89d40ea54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:10.284175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4abeae4c-a7bd-4b11-8e6f-bbf89d40ea54 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:10.285676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4abeae4c-a7bd-4b11-8e6f-bbf89d40ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:10.334235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:10.357092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1237/2486] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:33:10.365972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fff7d9-ad04-4f90-acb1-6bb14fbe8bd3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:10.366284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fff7d9-ad04-4f90-acb1-6bb14fbe8bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:10.366556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fff7d9-ad04-4f90-acb1-6bb14fbe8bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:10.386870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-75fff7d9-ad04-4f90-acb1-6bb14fbe8bd3 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:10.388813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-75fff7d9-ad04-4f90-acb1-6bb14fbe8bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:10.401281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1250/2487] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:10.405753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abeae4c-a7bd-4b11-8e6f-bbf89d40ea54 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:10.415667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abeae4c-a7bd-4b11-8e6f-bbf89d40ea54 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:10.420602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d15f1-819e-4a80-bd89-f28b7b585fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:10.421423 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d15f1-819e-4a80-bd89-f28b7b585fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:10.421792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d15f1-819e-4a80-bd89-f28b7b585fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:10.422173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d15f1-819e-4a80-bd89-f28b7b585fb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:10.422902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d15f1-819e-4a80-bd89-f28b7b585fb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:33:10.423273 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d15f1-819e-4a80-bd89-f28b7b585fb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:10.424981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d15f1-819e-4a80-bd89-f28b7b585fb6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:10.437601 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1238/2488] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:33:10.442243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fff7d9-ad04-4f90-acb1-6bb14fbe8bd3 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:10.452264 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fff7d9-ad04-4f90-acb1-6bb14fbe8bd3 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:10.457436 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34132c-e9db-488d-bc13-ad466b446db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:10.458151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34132c-e9db-488d-bc13-ad466b446db3 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:10.458426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34132c-e9db-488d-bc13-ad466b446db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:10.458691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34132c-e9db-488d-bc13-ad466b446db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:10.459333 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34132c-e9db-488d-bc13-ad466b446db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:10.461236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34132c-e9db-488d-bc13-ad466b446db3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:10.497956 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1251/2489] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:33:10 2022] POST /identity/v3/projects => generated 355 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:10.504990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d15f1-819e-4a80-bd89-f28b7b585fb6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:10.517985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d15f1-819e-4a80-bd89-f28b7b585fb6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:10.593765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcff9da-67e9-45d1-b26c-b3d08b1a9eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:10.594593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcff9da-67e9-45d1-b26c-b3d08b1a9eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:10.595329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcff9da-67e9-45d1-b26c-b3d08b1a9eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:10.595329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcff9da-67e9-45d1-b26c-b3d08b1a9eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:10.595765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcff9da-67e9-45d1-b26c-b3d08b1a9eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:10.597805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcff9da-67e9-45d1-b26c-b3d08b1a9eb8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:10.623350 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1239/2490] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:33:10 2022] POST /identity/v3/users => generated 278 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:10.627541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34132c-e9db-488d-bc13-ad466b446db3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:10.638306 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34132c-e9db-488d-bc13-ad466b446db3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:10.645143 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b85a0-f771-4e40-a623-9aed0d3dde31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:10.645989 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b85a0-f771-4e40-a623-9aed0d3dde31 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:10.646311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b85a0-f771-4e40-a623-9aed0d3dde31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:10.646582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b85a0-f771-4e40-a623-9aed0d3dde31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:10.648252 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b85a0-f771-4e40-a623-9aed0d3dde31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:10.649701 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893b85a0-f771-4e40-a623-9aed0d3dde31 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:10.664375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1252/2491] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:10.665252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcff9da-67e9-45d1-b26c-b3d08b1a9eb8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:10.675941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcff9da-67e9-45d1-b26c-b3d08b1a9eb8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:10.693855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a1230-9cbe-418c-b88f-e9a3e74966d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:10.695497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a1230-9cbe-418c-b88f-e9a3e74966d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:10.696080 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a1230-9cbe-418c-b88f-e9a3e74966d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:10.696435 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a1230-9cbe-418c-b88f-e9a3e74966d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:10.697292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a1230-9cbe-418c-b88f-e9a3e74966d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:10.700648 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a1230-9cbe-418c-b88f-e9a3e74966d8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:10.726284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1240/2492] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:33:10.730630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b85a0-f771-4e40-a623-9aed0d3dde31 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:10.741795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b85a0-f771-4e40-a623-9aed0d3dde31 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:10.757379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb9494-0cec-4f59-83df-b70b07e4bc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:10.758363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb9494-0cec-4f59-83df-b70b07e4bc7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:10.758966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb9494-0cec-4f59-83df-b70b07e4bc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:10.759386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb9494-0cec-4f59-83df-b70b07e4bc7b admin admin] PATH_INFO: `/v3/projects/729b4a873885495bbafd5d83e920944b/users/3e54d0b50d10470cafe2bbc76a0de80c/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:10.775743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb9494-0cec-4f59-83df-b70b07e4bc7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=729b4a873885495bbafd5d83e920944b, user_id=3e54d0b50d10470cafe2bbc76a0de80c, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:10.778898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb9494-0cec-4f59-83df-b70b07e4bc7b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:10.798591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1253/2493] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:10 2022] PUT /identity/v3/projects/729b4a873885495bbafd5d83e920944b/users/3e54d0b50d10470cafe2bbc76a0de80c/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:10.808800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889a919-ebb8-414e-85e9-c6ef9be878df None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:10.809349 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889a919-ebb8-414e-85e9-c6ef9be878df None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:10.809349 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889a919-ebb8-414e-85e9-c6ef9be878df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:10.860166 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e889a919-ebb8-414e-85e9-c6ef9be878df None None] MFA Rules not processed for user `3e54d0b50d10470cafe2bbc76a0de80c`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:10.861411 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e889a919-ebb8-414e-85e9-c6ef9be878df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:10.929969 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1241/2494] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:10.934451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb9494-0cec-4f59-83df-b70b07e4bc7b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:10.994729 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb9494-0cec-4f59-83df-b70b07e4bc7b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:10.999403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e52860-478d-4435-9724-597dfb796cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:10.999990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e52860-478d-4435-9724-597dfb796cad admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:11.000260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e52860-478d-4435-9724-597dfb796cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:11.000526 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e52860-478d-4435-9724-597dfb796cad admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:11.001152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e52860-478d-4435-9724-597dfb796cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:11.003073 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e52860-478d-4435-9724-597dfb796cad admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:11.029403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1254/2495] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:33:10 2022] POST /identity/v3/projects => generated 355 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:33:11.033572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889a919-ebb8-414e-85e9-c6ef9be878df None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:11.044229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889a919-ebb8-414e-85e9-c6ef9be878df None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:11.083746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:11.101503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a0802-9ad5-4e16-be8a-8677f7d67286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:11.102120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a0802-9ad5-4e16-be8a-8677f7d67286 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:11.102397 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a0802-9ad5-4e16-be8a-8677f7d67286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:11.102665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a0802-9ad5-4e16-be8a-8677f7d67286 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:11.103284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a0802-9ad5-4e16-be8a-8677f7d67286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:11.105202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a0802-9ad5-4e16-be8a-8677f7d67286 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:11.126060 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1242/2496] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:33:11 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:11.129976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e52860-478d-4435-9724-597dfb796cad admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:11.140662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e52860-478d-4435-9724-597dfb796cad admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:11.145283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d526e27-13e0-4171-95ab-a2b1974a59ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:11.145865 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d526e27-13e0-4171-95ab-a2b1974a59ae admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:11.146165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d526e27-13e0-4171-95ab-a2b1974a59ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:11.146433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d526e27-13e0-4171-95ab-a2b1974a59ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:11.147078 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d526e27-13e0-4171-95ab-a2b1974a59ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:11.148649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d526e27-13e0-4171-95ab-a2b1974a59ae admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:11.160389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1255/2497] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:11.164375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a0802-9ad5-4e16-be8a-8677f7d67286 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:11.174315 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a0802-9ad5-4e16-be8a-8677f7d67286 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:11.178948 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c9811-4126-4dc7-b47a-97de1c70478e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:11.179583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c9811-4126-4dc7-b47a-97de1c70478e admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:11.179861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c9811-4126-4dc7-b47a-97de1c70478e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:11.180133 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7c9811-4126-4dc7-b47a-97de1c70478e admin admin] PATH_INFO: `/v3/projects/94fcbc3c38f745689310dd6e59274ffa/users/f3689c5388744e828f42d370b78e9523/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:11.195343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c9811-4126-4dc7-b47a-97de1c70478e admin admin] RBAC: Authorizing `identity:create_grant(project_id=94fcbc3c38f745689310dd6e59274ffa, user_id=f3689c5388744e828f42d370b78e9523, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:11.198046 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7c9811-4126-4dc7-b47a-97de1c70478e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:11.211765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1243/2498] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:11 2022] PUT /identity/v3/projects/94fcbc3c38f745689310dd6e59274ffa/users/f3689c5388744e828f42d370b78e9523/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:11.216342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d526e27-13e0-4171-95ab-a2b1974a59ae admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:11.226259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d526e27-13e0-4171-95ab-a2b1974a59ae admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:11.263835 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:11.283822 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c6bfd-4931-43c8-b9f4-b31a36a58569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:11.284414 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c6bfd-4931-43c8-b9f4-b31a36a58569 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:11.284691 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c6bfd-4931-43c8-b9f4-b31a36a58569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:11.284959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c6bfd-4931-43c8-b9f4-b31a36a58569 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:11.285596 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c6bfd-4931-43c8-b9f4-b31a36a58569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:11.287248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c6bfd-4931-43c8-b9f4-b31a36a58569 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:11.296786 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1256/2499] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:11.301353 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c9811-4126-4dc7-b47a-97de1c70478e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:11.312052 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7c9811-4126-4dc7-b47a-97de1c70478e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:11.316632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e540f0-5bdd-440c-94b7-a7be208c542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:11.317267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e540f0-5bdd-440c-94b7-a7be208c542f admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:11.317539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e540f0-5bdd-440c-94b7-a7be208c542f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:11.317809 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e540f0-5bdd-440c-94b7-a7be208c542f admin admin] PATH_INFO: `/v3/projects/94fcbc3c38f745689310dd6e59274ffa/users/f3689c5388744e828f42d370b78e9523/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:11.319752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e540f0-5bdd-440c-94b7-a7be208c542f admin admin] RBAC: Authorizing `identity:create_grant(project_id=94fcbc3c38f745689310dd6e59274ffa, user_id=f3689c5388744e828f42d370b78e9523, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:11.322401 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e540f0-5bdd-440c-94b7-a7be208c542f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:11.333013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1244/2500] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:11 2022] PUT /identity/v3/projects/94fcbc3c38f745689310dd6e59274ffa/users/f3689c5388744e828f42d370b78e9523/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:11.343240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eac67-82f5-4ec4-b1d8-df5638cd9c13 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:11.343522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eac67-82f5-4ec4-b1d8-df5638cd9c13 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:11.343791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7eac67-82f5-4ec4-b1d8-df5638cd9c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:11.370902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-4d7eac67-82f5-4ec4-b1d8-df5638cd9c13 None None] MFA Rules not processed for user `f3689c5388744e828f42d370b78e9523`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:11.372599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4d7eac67-82f5-4ec4-b1d8-df5638cd9c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:11.443619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1257/2501] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:11 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:11.455505 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e540f0-5bdd-440c-94b7-a7be208c542f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:11.457184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-84e540f0-5bdd-440c-94b7-a7be208c542f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:11.534485 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e540f0-5bdd-440c-94b7-a7be208c542f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:11.538194 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f323120-8ba9-4c7a-b529-d5cef863f92e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:11.538812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f323120-8ba9-4c7a-b529-d5cef863f92e service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:11.539088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f323120-8ba9-4c7a-b529-d5cef863f92e service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:11.539359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f323120-8ba9-4c7a-b529-d5cef863f92e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:11.548690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f323120-8ba9-4c7a-b529-d5cef863f92e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:11.550121 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f323120-8ba9-4c7a-b529-d5cef863f92e service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:11.559972 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1245/2502] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:11.634988 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eac67-82f5-4ec4-b1d8-df5638cd9c13 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:11.644614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7eac67-82f5-4ec4-b1d8-df5638cd9c13 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:11.648184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dcb54-41e5-4df9-bf94-83a9d4d984dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:11.648776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dcb54-41e5-4df9-bf94-83a9d4d984dc service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:11.649049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dcb54-41e5-4df9-bf94-83a9d4d984dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:11.649317 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dcb54-41e5-4df9-bf94-83a9d4d984dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:11.705311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3dcb54-41e5-4df9-bf94-83a9d4d984dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:11.706744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3dcb54-41e5-4df9-bf94-83a9d4d984dc service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:11.717860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1258/2503] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:21.555304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=60,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:21.712408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=73 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:26.696667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f323120-8ba9-4c7a-b529-d5cef863f92e service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:26.757692 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f323120-8ba9-4c7a-b529-d5cef863f92e service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:26.762332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:26.762987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:26.763265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:26.764168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] PATH_INFO: `/v3/users/3e54d0b50d10470cafe2bbc76a0de80c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:26.765320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e54d0b50d10470cafe2bbc76a0de80c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:26.767332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:26.817010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e54d0b50d10470cafe2bbc76a0de80c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:26.817671 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e54d0b50d10470cafe2bbc76a0de80c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:26.824380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e54d0b50d10470cafe2bbc76a0de80c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:26.831733 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e54d0b50d10470cafe2bbc76a0de80c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:26.843457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e54d0b50d10470cafe2bbc76a0de80c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:26.849208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1246/2504] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:33:26 2022] DELETE /identity/v3/users/3e54d0b50d10470cafe2bbc76a0de80c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:26.853658 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6e7c9-709d-45a0-9050-88e19865dd47 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:26.854152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6e7c9-709d-45a0-9050-88e19865dd47 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:26.854267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6e7c9-709d-45a0-9050-88e19865dd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:26.873799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-65e6e7c9-709d-45a0-9050-88e19865dd47 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:26.874943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-65e6e7c9-709d-45a0-9050-88e19865dd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:26.931319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1259/2505] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:26.941539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:26.942966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:26.999308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917552b-5197-4a71-9c04-cf6fc5ce4a47 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:27.002981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd3c6c-167a-49a8-83f0-66d056f7257c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:27.003577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd3c6c-167a-49a8-83f0-66d056f7257c service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:27.003850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd3c6c-167a-49a8-83f0-66d056f7257c service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:27.004122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd3c6c-167a-49a8-83f0-66d056f7257c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:27.012873 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd3c6c-167a-49a8-83f0-66d056f7257c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:27.014302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd3c6c-167a-49a8-83f0-66d056f7257c service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:27.023458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1247/2506] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:27.300019 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6e7c9-709d-45a0-9050-88e19865dd47 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:27.302348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-65e6e7c9-709d-45a0-9050-88e19865dd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:27.317826 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6e7c9-709d-45a0-9050-88e19865dd47 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:27.322651 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:27.323309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:27.323573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:27.323845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] PATH_INFO: `/v3/projects/729b4a873885495bbafd5d83e920944b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:27.324983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=729b4a873885495bbafd5d83e920944b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:27.326939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:27.374789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '729b4a873885495bbafd5d83e920944b', 'request_id': 'req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.380277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '729b4a873885495bbafd5d83e920944b', 'request_id': 'req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.386371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '729b4a873885495bbafd5d83e920944b', 'request_id': 'req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.398473 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] The token cache is being invalidate because project 729b4a873885495bbafd5d83e920944b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:33:27.398759 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.399422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.400905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1260/2507] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:33:27 2022] DELETE /identity/v3/projects/729b4a873885495bbafd5d83e920944b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:27.405501 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd3c6c-167a-49a8-83f0-66d056f7257c service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:27.415035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd3c6c-167a-49a8-83f0-66d056f7257c service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:27.433189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:27.475961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea94851-174b-447b-9866-40c820f84406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:27.476675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea94851-174b-447b-9866-40c820f84406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:27.476956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea94851-174b-447b-9866-40c820f84406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:27.477226 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea94851-174b-447b-9866-40c820f84406 admin admin] PATH_INFO: `/v3/users/f3689c5388744e828f42d370b78e9523` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:27.478390 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea94851-174b-447b-9866-40c820f84406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3689c5388744e828f42d370b78e9523)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:27.480292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea94851-174b-447b-9866-40c820f84406 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:27.526261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3ea94851-174b-447b-9866-40c820f84406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3689c5388744e828f42d370b78e9523'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.526971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3ea94851-174b-447b-9866-40c820f84406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3689c5388744e828f42d370b78e9523'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.534431 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3ea94851-174b-447b-9866-40c820f84406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3689c5388744e828f42d370b78e9523'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.541860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3ea94851-174b-447b-9866-40c820f84406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3689c5388744e828f42d370b78e9523'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.554411 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3ea94851-174b-447b-9866-40c820f84406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3689c5388744e828f42d370b78e9523'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.560151 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1248/2508] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:33:27 2022] DELETE /identity/v3/users/f3689c5388744e828f42d370b78e9523 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:27.834607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:27.836283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:27.851990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be9e2f-1ae9-414d-8167-4fa818b6c6a3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:27.908743 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596087c3-815a-4f3b-9acf-03a67de6812e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:27.909362 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596087c3-815a-4f3b-9acf-03a67de6812e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:27.909543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596087c3-815a-4f3b-9acf-03a67de6812e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:27.909811 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596087c3-815a-4f3b-9acf-03a67de6812e admin admin] PATH_INFO: `/v3/projects/94fcbc3c38f745689310dd6e59274ffa` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:27.910934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596087c3-815a-4f3b-9acf-03a67de6812e admin admin] RBAC: Authorizing `identity:delete_project(project_id=94fcbc3c38f745689310dd6e59274ffa)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:27.912856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596087c3-815a-4f3b-9acf-03a67de6812e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:27.942933 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:27.961086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-596087c3-815a-4f3b-9acf-03a67de6812e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94fcbc3c38f745689310dd6e59274ffa', 'request_id': 'req-596087c3-815a-4f3b-9acf-03a67de6812e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.967285 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-596087c3-815a-4f3b-9acf-03a67de6812e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94fcbc3c38f745689310dd6e59274ffa', 'request_id': 'req-596087c3-815a-4f3b-9acf-03a67de6812e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.973608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-596087c3-815a-4f3b-9acf-03a67de6812e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94fcbc3c38f745689310dd6e59274ffa', 'request_id': 'req-596087c3-815a-4f3b-9acf-03a67de6812e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.985979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-596087c3-815a-4f3b-9acf-03a67de6812e admin admin] The token cache is being invalidate because project 94fcbc3c38f745689310dd6e59274ffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:33:27.986293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-596087c3-815a-4f3b-9acf-03a67de6812e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.986923 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-596087c3-815a-4f3b-9acf-03a67de6812e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:27.988412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1261/2509] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:33:27 2022] DELETE /identity/v3/projects/94fcbc3c38f745689310dd6e59274ffa => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:27.993985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2351117-202a-425a-a6a7-9d9866a93f3a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:27.994476 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2351117-202a-425a-a6a7-9d9866a93f3a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:27.994572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2351117-202a-425a-a6a7-9d9866a93f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.015160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d2351117-202a-425a-a6a7-9d9866a93f3a None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:28.016367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d2351117-202a-425a-a6a7-9d9866a93f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:28.073305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1249/2510] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:33:28.082562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39f3b8-9184-4a16-a325-b274698f94fa None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:28.082848 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39f3b8-9184-4a16-a325-b274698f94fa None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:28.083114 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39f3b8-9184-4a16-a325-b274698f94fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.103932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-cf39f3b8-9184-4a16-a325-b274698f94fa None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:28.105124 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cf39f3b8-9184-4a16-a325-b274698f94fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:28.113285 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1262/2511] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:28.117657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2351117-202a-425a-a6a7-9d9866a93f3a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:28.127874 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2351117-202a-425a-a6a7-9d9866a93f3a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:28.132431 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2b36d-8b4f-425b-a3f9-5743bd6425f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:28.133016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2b36d-8b4f-425b-a3f9-5743bd6425f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:28.133292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2b36d-8b4f-425b-a3f9-5743bd6425f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:28.133565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2b36d-8b4f-425b-a3f9-5743bd6425f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.134245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2b36d-8b4f-425b-a3f9-5743bd6425f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:33:28.134550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2b36d-8b4f-425b-a3f9-5743bd6425f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:28.136137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d2b36d-8b4f-425b-a3f9-5743bd6425f2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:28.148023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1250/2512] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:33:28.152178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39f3b8-9184-4a16-a325-b274698f94fa None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:28.161211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39f3b8-9184-4a16-a325-b274698f94fa None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:28.165866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ce702-40a6-4fee-9caa-015a0e70a2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:28.166480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce702-40a6-4fee-9caa-015a0e70a2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:28.166753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce702-40a6-4fee-9caa-015a0e70a2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:28.167019 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce702-40a6-4fee-9caa-015a0e70a2a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.167647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ce702-40a6-4fee-9caa-015a0e70a2a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:28.169520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ce702-40a6-4fee-9caa-015a0e70a2a9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:28.197950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1263/2513] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:33:28 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:28.202778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2b36d-8b4f-425b-a3f9-5743bd6425f2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:28.212452 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d2b36d-8b4f-425b-a3f9-5743bd6425f2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:28.237608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:28.268713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea718ec-4efd-4108-8d93-779022b94d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:28.269306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea718ec-4efd-4108-8d93-779022b94d2a admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:28.269582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea718ec-4efd-4108-8d93-779022b94d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:28.269849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea718ec-4efd-4108-8d93-779022b94d2a admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.270516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea718ec-4efd-4108-8d93-779022b94d2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:28.272406 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea718ec-4efd-4108-8d93-779022b94d2a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:28.292004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1251/2514] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:33:28 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:33:28.296134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ce702-40a6-4fee-9caa-015a0e70a2a9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:28.307268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ce702-40a6-4fee-9caa-015a0e70a2a9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:28.312905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f605b-e968-41a3-ace1-fdf8a762ad9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:28.313501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f605b-e968-41a3-ace1-fdf8a762ad9e admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:28.313807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f605b-e968-41a3-ace1-fdf8a762ad9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:28.314098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f605b-e968-41a3-ace1-fdf8a762ad9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.314772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f605b-e968-41a3-ace1-fdf8a762ad9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:28.316370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f605b-e968-41a3-ace1-fdf8a762ad9e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:28.327111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1264/2515] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:28.331117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea718ec-4efd-4108-8d93-779022b94d2a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:28.341181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea718ec-4efd-4108-8d93-779022b94d2a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:28.345769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85a351-63fb-434a-a8db-416e7020442c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:28.346382 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85a351-63fb-434a-a8db-416e7020442c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:28.346659 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85a351-63fb-434a-a8db-416e7020442c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:28.346927 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85a351-63fb-434a-a8db-416e7020442c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.347560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85a351-63fb-434a-a8db-416e7020442c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:28.349134 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85a351-63fb-434a-a8db-416e7020442c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:28.360362 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1252/2516] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:28.364130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f605b-e968-41a3-ace1-fdf8a762ad9e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:28.374555 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f605b-e968-41a3-ace1-fdf8a762ad9e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:28.379072 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5a509-e3f6-409a-9658-73253f5dc67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:28.379705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a509-e3f6-409a-9658-73253f5dc67d admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:28.379980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a509-e3f6-409a-9658-73253f5dc67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:28.380248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a509-e3f6-409a-9658-73253f5dc67d admin admin] PATH_INFO: `/v3/projects/4b39565fbf9b4cbb954cb43f0934d90f/users/4503704eacd7432a8e786edbccf7861b/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.514502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5a509-e3f6-409a-9658-73253f5dc67d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b39565fbf9b4cbb954cb43f0934d90f, user_id=4503704eacd7432a8e786edbccf7861b, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:28.517030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5a509-e3f6-409a-9658-73253f5dc67d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:28.530220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1265/2517] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:28 2022] PUT /identity/v3/projects/4b39565fbf9b4cbb954cb43f0934d90f/users/4503704eacd7432a8e786edbccf7861b/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:28.540445 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b877bd-b44e-4273-bdb7-536660115882 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:28.540726 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b877bd-b44e-4273-bdb7-536660115882 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:28.541003 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b877bd-b44e-4273-bdb7-536660115882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.569129 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-13b877bd-b44e-4273-bdb7-536660115882 None None] MFA Rules not processed for user `4503704eacd7432a8e786edbccf7861b`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:28.570361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-13b877bd-b44e-4273-bdb7-536660115882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:28.633067 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1253/2518] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:28.637647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5a509-e3f6-409a-9658-73253f5dc67d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:28.697294 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5a509-e3f6-409a-9658-73253f5dc67d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:28.701979 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d278b8-59d1-4441-b3db-9d46e5c0483d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:28.702615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d278b8-59d1-4441-b3db-9d46e5c0483d admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:28.702896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d278b8-59d1-4441-b3db-9d46e5c0483d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:28.703159 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d278b8-59d1-4441-b3db-9d46e5c0483d admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.703795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d278b8-59d1-4441-b3db-9d46e5c0483d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:28.705693 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d278b8-59d1-4441-b3db-9d46e5c0483d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:28.733591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1266/2519] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:33:28 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:28.737724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b877bd-b44e-4273-bdb7-536660115882 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:28.748283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b877bd-b44e-4273-bdb7-536660115882 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:28.803043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa10bc0-c533-44fd-8adf-49f39b6edd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:28.803567 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa10bc0-c533-44fd-8adf-49f39b6edd25 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:28.803844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa10bc0-c533-44fd-8adf-49f39b6edd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:28.804115 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa10bc0-c533-44fd-8adf-49f39b6edd25 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.804752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa10bc0-c533-44fd-8adf-49f39b6edd25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:28.806694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa10bc0-c533-44fd-8adf-49f39b6edd25 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:28.827367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1254/2520] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:33:28 2022] POST /identity/v3/users => generated 300 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:33:28.831252 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d278b8-59d1-4441-b3db-9d46e5c0483d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:28.841200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d278b8-59d1-4441-b3db-9d46e5c0483d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:28.846606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806e696-6863-4812-91db-22ee3699b032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:28.847199 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806e696-6863-4812-91db-22ee3699b032 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:28.847476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806e696-6863-4812-91db-22ee3699b032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:28.847746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806e696-6863-4812-91db-22ee3699b032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.848394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806e696-6863-4812-91db-22ee3699b032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:28.850000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806e696-6863-4812-91db-22ee3699b032 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:28.860641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1267/2521] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:28.864415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa10bc0-c533-44fd-8adf-49f39b6edd25 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:28.874144 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa10bc0-c533-44fd-8adf-49f39b6edd25 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:28.879039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c404af9-a6bf-4716-ae65-1e86b18373ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:28.879680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c404af9-a6bf-4716-ae65-1e86b18373ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:28.879957 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c404af9-a6bf-4716-ae65-1e86b18373ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:28.880230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c404af9-a6bf-4716-ae65-1e86b18373ca admin admin] PATH_INFO: `/v3/projects/5d7552e031604ddfb6b30d5a8801f327/users/e0865b81e18c490a9901504acae735ab/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.894833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c404af9-a6bf-4716-ae65-1e86b18373ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d7552e031604ddfb6b30d5a8801f327, user_id=e0865b81e18c490a9901504acae735ab, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:28.897469 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c404af9-a6bf-4716-ae65-1e86b18373ca admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:28.911303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1255/2522] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:28 2022] PUT /identity/v3/projects/5d7552e031604ddfb6b30d5a8801f327/users/e0865b81e18c490a9901504acae735ab/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:28.915211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806e696-6863-4812-91db-22ee3699b032 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:28.928141 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806e696-6863-4812-91db-22ee3699b032 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:28.947679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:28.985984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c00e82-69b8-4c52-86f5-e8e0ebbadbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:28.986611 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c00e82-69b8-4c52-86f5-e8e0ebbadbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:28.986888 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c00e82-69b8-4c52-86f5-e8e0ebbadbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:28.987193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c00e82-69b8-4c52-86f5-e8e0ebbadbd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:28.987823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c00e82-69b8-4c52-86f5-e8e0ebbadbd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:28.989417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c00e82-69b8-4c52-86f5-e8e0ebbadbd3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:28.998901 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1268/2523] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:29.003003 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c404af9-a6bf-4716-ae65-1e86b18373ca admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:29.008402 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:29.014615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c404af9-a6bf-4716-ae65-1e86b18373ca admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:29.019657 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b22966-bba5-4a05-a948-b607bf1e4ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:29.020295 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b22966-bba5-4a05-a948-b607bf1e4ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:29.020578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b22966-bba5-4a05-a948-b607bf1e4ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:29.020851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b22966-bba5-4a05-a948-b607bf1e4ddd admin admin] PATH_INFO: `/v3/projects/5d7552e031604ddfb6b30d5a8801f327/users/e0865b81e18c490a9901504acae735ab/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:29.022838 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b22966-bba5-4a05-a948-b607bf1e4ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d7552e031604ddfb6b30d5a8801f327, user_id=e0865b81e18c490a9901504acae735ab, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:29.025509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b22966-bba5-4a05-a948-b607bf1e4ddd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:29.036022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1256/2524] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:29 2022] PUT /identity/v3/projects/5d7552e031604ddfb6b30d5a8801f327/users/e0865b81e18c490a9901504acae735ab/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:29.047717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e52e4-77b3-4368-abfe-023f2a1e5688 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:29.048000 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e52e4-77b3-4368-abfe-023f2a1e5688 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:29.048273 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e52e4-77b3-4368-abfe-023f2a1e5688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:29.075289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c16e52e4-77b3-4368-abfe-023f2a1e5688 None None] MFA Rules not processed for user `e0865b81e18c490a9901504acae735ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:29.076915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c16e52e4-77b3-4368-abfe-023f2a1e5688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:29.147168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1269/2525] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:29.158641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b22966-bba5-4a05-a948-b607bf1e4ddd admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:29.160354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-95b22966-bba5-4a05-a948-b607bf1e4ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:29.218822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b22966-bba5-4a05-a948-b607bf1e4ddd admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:29.222464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcb605-b56e-454d-904e-f54f9ad70a86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:29.223058 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcb605-b56e-454d-904e-f54f9ad70a86 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:29.223338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcb605-b56e-454d-904e-f54f9ad70a86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:29.223607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcb605-b56e-454d-904e-f54f9ad70a86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:29.232785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcb605-b56e-454d-904e-f54f9ad70a86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:29.234245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcb605-b56e-454d-904e-f54f9ad70a86 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:29.243580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1257/2526] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:29.377488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e52e4-77b3-4368-abfe-023f2a1e5688 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:29.443191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e52e4-77b3-4368-abfe-023f2a1e5688 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:29.447662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:29.448287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:29.448563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:29.448833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] PATH_INFO: `/v3/users/4503704eacd7432a8e786edbccf7861b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:29.449954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4503704eacd7432a8e786edbccf7861b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:29.452447 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:29.498790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4503704eacd7432a8e786edbccf7861b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:29.499929 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:29.508258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4503704eacd7432a8e786edbccf7861b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:29.508913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4503704eacd7432a8e786edbccf7861b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:29.516748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4503704eacd7432a8e786edbccf7861b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:29.529259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4503704eacd7432a8e786edbccf7861b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:29.535457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1270/2527] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:33:29 2022] DELETE /identity/v3/users/4503704eacd7432a8e786edbccf7861b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:29.539752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1f136-9e99-4e37-8ced-e0bcf4933dcd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:29.540034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1f136-9e99-4e37-8ced-e0bcf4933dcd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:29.540305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1f136-9e99-4e37-8ced-e0bcf4933dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:29.558627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8be1f136-9e99-4e37-8ced-e0bcf4933dcd None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:29.559816 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8be1f136-9e99-4e37-8ced-e0bcf4933dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:29.617274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1258/2528] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:29.627537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:29.628944 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:29.684661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62317de4-2886-44b6-97de-624310d83541 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:29.688770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd798248-fba4-474e-a606-2b8052afde8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:29.689367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd798248-fba4-474e-a606-2b8052afde8e service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:29.689644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd798248-fba4-474e-a606-2b8052afde8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:29.689915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd798248-fba4-474e-a606-2b8052afde8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:29.698689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd798248-fba4-474e-a606-2b8052afde8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:29.700110 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd798248-fba4-474e-a606-2b8052afde8e service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:29.710469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1271/2529] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:29.975688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1f136-9e99-4e37-8ced-e0bcf4933dcd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:29.977324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8be1f136-9e99-4e37-8ced-e0bcf4933dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:29.990201 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1f136-9e99-4e37-8ced-e0bcf4933dcd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:29.994797 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:29.995430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:29.995706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:29.995981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] PATH_INFO: `/v3/projects/4b39565fbf9b4cbb954cb43f0934d90f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:29.997148 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b39565fbf9b4cbb954cb43f0934d90f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:29.999776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:30.046971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b39565fbf9b4cbb954cb43f0934d90f', 'request_id': 'req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.052456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b39565fbf9b4cbb954cb43f0934d90f', 'request_id': 'req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.058670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b39565fbf9b4cbb954cb43f0934d90f', 'request_id': 'req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.071416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] The token cache is being invalidate because project 4b39565fbf9b4cbb954cb43f0934d90f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:33:30.071700 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.072381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.073889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1259/2530] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:33:29 2022] DELETE /identity/v3/projects/4b39565fbf9b4cbb954cb43f0934d90f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:33:30.078436 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd798248-fba4-474e-a606-2b8052afde8e service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:30.087982 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd798248-fba4-474e-a606-2b8052afde8e service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:30.142905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c66df9-b60a-49a6-9714-cb1109e6e0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:30.143520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c66df9-b60a-49a6-9714-cb1109e6e0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:30.143800 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c66df9-b60a-49a6-9714-cb1109e6e0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:30.144068 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c66df9-b60a-49a6-9714-cb1109e6e0f9 admin admin] PATH_INFO: `/v3/users/e0865b81e18c490a9901504acae735ab` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:30.145201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c66df9-b60a-49a6-9714-cb1109e6e0f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0865b81e18c490a9901504acae735ab)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:30.147154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c66df9-b60a-49a6-9714-cb1109e6e0f9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:30.192715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-53c66df9-b60a-49a6-9714-cb1109e6e0f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0865b81e18c490a9901504acae735ab'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.199400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-53c66df9-b60a-49a6-9714-cb1109e6e0f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0865b81e18c490a9901504acae735ab'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.200044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-53c66df9-b60a-49a6-9714-cb1109e6e0f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0865b81e18c490a9901504acae735ab'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.207791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-53c66df9-b60a-49a6-9714-cb1109e6e0f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0865b81e18c490a9901504acae735ab'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.212826 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:30.221463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-53c66df9-b60a-49a6-9714-cb1109e6e0f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0865b81e18c490a9901504acae735ab'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.227356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1272/2531] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:33:30 2022] DELETE /identity/v3/users/e0865b81e18c490a9901504acae735ab => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:30.493627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:30.495132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:30.509249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98b4b7-d4e2-4b54-8062-a371c20c12f0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:30.523073 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:30.569520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfabf3e-2d43-4831-ab53-c176d043ae38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:30.570172 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfabf3e-2d43-4831-ab53-c176d043ae38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:30.570451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfabf3e-2d43-4831-ab53-c176d043ae38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:30.570723 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfabf3e-2d43-4831-ab53-c176d043ae38 admin admin] PATH_INFO: `/v3/projects/5d7552e031604ddfb6b30d5a8801f327` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:30.571861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfabf3e-2d43-4831-ab53-c176d043ae38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d7552e031604ddfb6b30d5a8801f327)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:30.573804 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfabf3e-2d43-4831-ab53-c176d043ae38 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:30.624399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cdfabf3e-2d43-4831-ab53-c176d043ae38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d7552e031604ddfb6b30d5a8801f327', 'request_id': 'req-cdfabf3e-2d43-4831-ab53-c176d043ae38'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.629769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cdfabf3e-2d43-4831-ab53-c176d043ae38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d7552e031604ddfb6b30d5a8801f327', 'request_id': 'req-cdfabf3e-2d43-4831-ab53-c176d043ae38'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.635863 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cdfabf3e-2d43-4831-ab53-c176d043ae38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d7552e031604ddfb6b30d5a8801f327', 'request_id': 'req-cdfabf3e-2d43-4831-ab53-c176d043ae38'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.648643 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cdfabf3e-2d43-4831-ab53-c176d043ae38 admin admin] The token cache is being invalidate because project 5d7552e031604ddfb6b30d5a8801f327 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:33:30.649063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cdfabf3e-2d43-4831-ab53-c176d043ae38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.649676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cdfabf3e-2d43-4831-ab53-c176d043ae38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:30.651198 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1260/2532] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:33:30 2022] DELETE /identity/v3/projects/5d7552e031604ddfb6b30d5a8801f327 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:30.656754 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e10fe-6b39-4347-bd29-6d4ab6da5553 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:30.657030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e10fe-6b39-4347-bd29-6d4ab6da5553 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:30.657294 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e10fe-6b39-4347-bd29-6d4ab6da5553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:30.676093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-415e10fe-6b39-4347-bd29-6d4ab6da5553 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:30.677265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-415e10fe-6b39-4347-bd29-6d4ab6da5553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:30.734719 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1273/2533] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:30.742584 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b801d3-48d1-420b-bbe1-2893c409584c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:30.742799 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b801d3-48d1-420b-bbe1-2893c409584c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:30.743070 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b801d3-48d1-420b-bbe1-2893c409584c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:30.762429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-98b801d3-48d1-420b-bbe1-2893c409584c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:30.764103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-98b801d3-48d1-420b-bbe1-2893c409584c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:30.771891 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1261/2534] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:30.776149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e10fe-6b39-4347-bd29-6d4ab6da5553 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:30.786323 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e10fe-6b39-4347-bd29-6d4ab6da5553 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:30.790916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc96ac2-b4a1-42d5-88f8-2193db9f67da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:30.791522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc96ac2-b4a1-42d5-88f8-2193db9f67da admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:30.791804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc96ac2-b4a1-42d5-88f8-2193db9f67da admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:30.792107 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc96ac2-b4a1-42d5-88f8-2193db9f67da admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:30.792761 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc96ac2-b4a1-42d5-88f8-2193db9f67da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:33:30.793061 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc96ac2-b4a1-42d5-88f8-2193db9f67da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:30.794745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc96ac2-b4a1-42d5-88f8-2193db9f67da admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:30.807561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1274/2535] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:33:30.810337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b801d3-48d1-420b-bbe1-2893c409584c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:30.819575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b801d3-48d1-420b-bbe1-2893c409584c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:30.824143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c41d4-7997-434f-9cfe-d134c6f4149d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:30.824719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c41d4-7997-434f-9cfe-d134c6f4149d admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:30.824989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c41d4-7997-434f-9cfe-d134c6f4149d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:30.825255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c41d4-7997-434f-9cfe-d134c6f4149d admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:30.826422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c41d4-7997-434f-9cfe-d134c6f4149d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:30.828286 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c41d4-7997-434f-9cfe-d134c6f4149d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:30.854963 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1262/2536] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:33:30 2022] POST /identity/v3/projects => generated 403 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:30.859400 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc96ac2-b4a1-42d5-88f8-2193db9f67da admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:30.869200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc96ac2-b4a1-42d5-88f8-2193db9f67da admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:30.923044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387cb26-ab02-402e-a071-39e8a98ac0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:30.923639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387cb26-ab02-402e-a071-39e8a98ac0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:30.923916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387cb26-ab02-402e-a071-39e8a98ac0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:30.924180 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387cb26-ab02-402e-a071-39e8a98ac0f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:30.924825 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387cb26-ab02-402e-a071-39e8a98ac0f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:30.926773 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387cb26-ab02-402e-a071-39e8a98ac0f3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:30.946675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1275/2537] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:33:30 2022] POST /identity/v3/users => generated 302 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:30.951430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c41d4-7997-434f-9cfe-d134c6f4149d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:30.961510 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c41d4-7997-434f-9cfe-d134c6f4149d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:30.966004 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7646ce8-f34f-4391-bdf6-e40dd69c2e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:30.967191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7646ce8-f34f-4391-bdf6-e40dd69c2e1c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:30.967462 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7646ce8-f34f-4391-bdf6-e40dd69c2e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:30.967733 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7646ce8-f34f-4391-bdf6-e40dd69c2e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:30.968358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7646ce8-f34f-4391-bdf6-e40dd69c2e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:30.969915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7646ce8-f34f-4391-bdf6-e40dd69c2e1c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:30.979615 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1263/2538] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:33:30.983604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387cb26-ab02-402e-a071-39e8a98ac0f3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:30.993229 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387cb26-ab02-402e-a071-39e8a98ac0f3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:30.998260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90deff-3b78-4206-8ea0-09e942c1226a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:30.998836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90deff-3b78-4206-8ea0-09e942c1226a admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:30.999109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90deff-3b78-4206-8ea0-09e942c1226a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:30.999375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90deff-3b78-4206-8ea0-09e942c1226a admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:31.000001 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90deff-3b78-4206-8ea0-09e942c1226a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:31.001562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90deff-3b78-4206-8ea0-09e942c1226a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:31.011764 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1276/2539] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:31.016247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7646ce8-f34f-4391-bdf6-e40dd69c2e1c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:31.026466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7646ce8-f34f-4391-bdf6-e40dd69c2e1c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:31.031024 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6577b-6a48-4b0f-8f80-f95fb3685ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:31.031645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6577b-6a48-4b0f-8f80-f95fb3685ab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:31.031918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6577b-6a48-4b0f-8f80-f95fb3685ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:31.032184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6577b-6a48-4b0f-8f80-f95fb3685ab1 admin admin] PATH_INFO: `/v3/projects/08cb3a4c593a42148a0441a19aff0b67/users/58c5f46caa6a4718b7f0abc6ff754be9/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:31.045998 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6577b-6a48-4b0f-8f80-f95fb3685ab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08cb3a4c593a42148a0441a19aff0b67, user_id=58c5f46caa6a4718b7f0abc6ff754be9, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:31.048634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6577b-6a48-4b0f-8f80-f95fb3685ab1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:31.061444 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1264/2540] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:31 2022] PUT /identity/v3/projects/08cb3a4c593a42148a0441a19aff0b67/users/58c5f46caa6a4718b7f0abc6ff754be9/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:31.071860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d0572-5d57-4493-bbbe-b1eef002caa5 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:31.072142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d0572-5d57-4493-bbbe-b1eef002caa5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:31.072409 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d0572-5d57-4493-bbbe-b1eef002caa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:31.099664 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d24d0572-5d57-4493-bbbe-b1eef002caa5 None None] MFA Rules not processed for user `58c5f46caa6a4718b7f0abc6ff754be9`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:31.100843 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d24d0572-5d57-4493-bbbe-b1eef002caa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:31.140939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:31.167728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1277/2541] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:31 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:31.168937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6577b-6a48-4b0f-8f80-f95fb3685ab1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:31.213427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:31.232272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6577b-6a48-4b0f-8f80-f95fb3685ab1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:31.238607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36225013-70df-4024-8edf-68f2b08f02cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:31.239204 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36225013-70df-4024-8edf-68f2b08f02cc admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:31.239490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36225013-70df-4024-8edf-68f2b08f02cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:31.239755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36225013-70df-4024-8edf-68f2b08f02cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:31.240384 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36225013-70df-4024-8edf-68f2b08f02cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:31.242300 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36225013-70df-4024-8edf-68f2b08f02cc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:31.270516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1265/2542] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:33:31 2022] POST /identity/v3/projects => generated 409 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:31.274674 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d0572-5d57-4493-bbbe-b1eef002caa5 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:31.285123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d0572-5d57-4493-bbbe-b1eef002caa5 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:31.340775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302b4a1-a33d-4dcd-a0c5-40cab00a5574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:31.341632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302b4a1-a33d-4dcd-a0c5-40cab00a5574 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:31.341753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302b4a1-a33d-4dcd-a0c5-40cab00a5574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:31.341926 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302b4a1-a33d-4dcd-a0c5-40cab00a5574 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:31.342587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302b4a1-a33d-4dcd-a0c5-40cab00a5574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:31.344500 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302b4a1-a33d-4dcd-a0c5-40cab00a5574 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:31.365411 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1278/2543] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:33:31 2022] POST /identity/v3/users => generated 311 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:31.369630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36225013-70df-4024-8edf-68f2b08f02cc admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:31.380556 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36225013-70df-4024-8edf-68f2b08f02cc admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:31.385186 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84620513-2c45-458f-95fd-c7d05623bb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:31.385772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84620513-2c45-458f-95fd-c7d05623bb01 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:31.386050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84620513-2c45-458f-95fd-c7d05623bb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:31.386360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84620513-2c45-458f-95fd-c7d05623bb01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:31.386998 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84620513-2c45-458f-95fd-c7d05623bb01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:31.388608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84620513-2c45-458f-95fd-c7d05623bb01 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:31.399020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1266/2544] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:31.402916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302b4a1-a33d-4dcd-a0c5-40cab00a5574 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:31.412468 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302b4a1-a33d-4dcd-a0c5-40cab00a5574 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:31.416972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d339ea7-5089-490a-ab58-a3765575042b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:31.417598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d339ea7-5089-490a-ab58-a3765575042b admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:31.417870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d339ea7-5089-490a-ab58-a3765575042b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:31.418171 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d339ea7-5089-490a-ab58-a3765575042b admin admin] PATH_INFO: `/v3/projects/c4aab6a8b4464312bd1e21a9b647e5fe/users/e6ba65f868ad487b81ee63191cc40b70/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:31.432939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d339ea7-5089-490a-ab58-a3765575042b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4aab6a8b4464312bd1e21a9b647e5fe, user_id=e6ba65f868ad487b81ee63191cc40b70, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:31.436373 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d339ea7-5089-490a-ab58-a3765575042b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:31.448905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1279/2545] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:31 2022] PUT /identity/v3/projects/c4aab6a8b4464312bd1e21a9b647e5fe/users/e6ba65f868ad487b81ee63191cc40b70/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:31.452790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84620513-2c45-458f-95fd-c7d05623bb01 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:31.462776 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84620513-2c45-458f-95fd-c7d05623bb01 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:31.518882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5a626-2876-4866-8dd3-25de18319e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:31.519476 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5a626-2876-4866-8dd3-25de18319e13 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:31.519746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5a626-2876-4866-8dd3-25de18319e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:31.520012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5a626-2876-4866-8dd3-25de18319e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:31.520689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5a626-2876-4866-8dd3-25de18319e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:31.522233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5a626-2876-4866-8dd3-25de18319e13 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:31.531856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1267/2546] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:31.536007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d339ea7-5089-490a-ab58-a3765575042b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:31.546623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d339ea7-5089-490a-ab58-a3765575042b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:31.551154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742d4dc-719c-4da6-8cff-3c173c6e3aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:31.551793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742d4dc-719c-4da6-8cff-3c173c6e3aee admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:31.552068 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742d4dc-719c-4da6-8cff-3c173c6e3aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:31.552341 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742d4dc-719c-4da6-8cff-3c173c6e3aee admin admin] PATH_INFO: `/v3/projects/c4aab6a8b4464312bd1e21a9b647e5fe/users/e6ba65f868ad487b81ee63191cc40b70/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:31.554288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742d4dc-719c-4da6-8cff-3c173c6e3aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4aab6a8b4464312bd1e21a9b647e5fe, user_id=e6ba65f868ad487b81ee63191cc40b70, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:31.556900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742d4dc-719c-4da6-8cff-3c173c6e3aee admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:31.567455 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1280/2547] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:31 2022] PUT /identity/v3/projects/c4aab6a8b4464312bd1e21a9b647e5fe/users/e6ba65f868ad487b81ee63191cc40b70/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:31.578516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53cff9-d016-478c-ab0d-eac512d70838 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:31.578797 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53cff9-d016-478c-ab0d-eac512d70838 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:31.579066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53cff9-d016-478c-ab0d-eac512d70838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:31.605712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-cd53cff9-d016-478c-ab0d-eac512d70838 None None] MFA Rules not processed for user `e6ba65f868ad487b81ee63191cc40b70`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:31.607703 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cd53cff9-d016-478c-ab0d-eac512d70838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:31.676561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1268/2548] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:31 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:31.687725 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742d4dc-719c-4da6-8cff-3c173c6e3aee admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:31.689451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6742d4dc-719c-4da6-8cff-3c173c6e3aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:31.746173 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742d4dc-719c-4da6-8cff-3c173c6e3aee admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:31.749831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa560b1-dee6-4ca7-8be3-08eb853f223d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:31.750465 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa560b1-dee6-4ca7-8be3-08eb853f223d service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:31.750741 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa560b1-dee6-4ca7-8be3-08eb853f223d service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:31.751009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa560b1-dee6-4ca7-8be3-08eb853f223d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:31.760012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa560b1-dee6-4ca7-8be3-08eb853f223d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:31.761437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa560b1-dee6-4ca7-8be3-08eb853f223d service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:31.770945 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1281/2549] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:31 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:31.919302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53cff9-d016-478c-ab0d-eac512d70838 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:31.929177 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53cff9-d016-478c-ab0d-eac512d70838 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:31.932649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda68c38-0187-45ab-a4f8-120b00c0d1c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:31.933229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda68c38-0187-45ab-a4f8-120b00c0d1c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:31.933502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda68c38-0187-45ab-a4f8-120b00c0d1c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:31.933769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda68c38-0187-45ab-a4f8-120b00c0d1c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:31.966811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:31.991391 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda68c38-0187-45ab-a4f8-120b00c0d1c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:31.992812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda68c38-0187-45ab-a4f8-120b00c0d1c1 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:32.001712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1269/2550] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:31 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:41.352121 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa560b1-dee6-4ca7-8be3-08eb853f223d service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:41.392017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:41.413992 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa560b1-dee6-4ca7-8be3-08eb853f223d service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:41.418484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:41.419105 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:41.419380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:41.419655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] PATH_INFO: `/v3/users/58c5f46caa6a4718b7f0abc6ff754be9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:41.420772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c5f46caa6a4718b7f0abc6ff754be9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:41.422723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:41.468280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c5f46caa6a4718b7f0abc6ff754be9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:41.474991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c5f46caa6a4718b7f0abc6ff754be9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:41.475634 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c5f46caa6a4718b7f0abc6ff754be9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:41.482969 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c5f46caa6a4718b7f0abc6ff754be9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:41.494991 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c5f46caa6a4718b7f0abc6ff754be9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:41.501962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1282/2551] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:33:41 2022] DELETE /identity/v3/users/58c5f46caa6a4718b7f0abc6ff754be9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:41.506437 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289f71d-826d-4d53-9f7b-4080977e3e04 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:41.506914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289f71d-826d-4d53-9f7b-4080977e3e04 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:41.507013 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289f71d-826d-4d53-9f7b-4080977e3e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:41.526048 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3289f71d-826d-4d53-9f7b-4080977e3e04 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:41.527253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3289f71d-826d-4d53-9f7b-4080977e3e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:41.584641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1270/2552] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:33:41.594527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:41.595921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:41.652034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dee643-4bf1-494a-84e3-54fa62ec482c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:41.655666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c8292-0cf7-403c-96cc-2599b3ace750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:41.656259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c8292-0cf7-403c-96cc-2599b3ace750 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:41.656534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c8292-0cf7-403c-96cc-2599b3ace750 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:41.656807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c8292-0cf7-403c-96cc-2599b3ace750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:41.665564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c8292-0cf7-403c-96cc-2599b3ace750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:41.667010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c8292-0cf7-403c-96cc-2599b3ace750 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:41.676243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1283/2553] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:41.940057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289f71d-826d-4d53-9f7b-4080977e3e04 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:41.941504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3289f71d-826d-4d53-9f7b-4080977e3e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:41.956876 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289f71d-826d-4d53-9f7b-4080977e3e04 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:41.962441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:41.962978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:41.963270 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:41.963530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] PATH_INFO: `/v3/projects/08cb3a4c593a42148a0441a19aff0b67` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:41.964649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08cb3a4c593a42148a0441a19aff0b67)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:41.966591 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:42.012722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08cb3a4c593a42148a0441a19aff0b67', 'request_id': 'req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.018219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08cb3a4c593a42148a0441a19aff0b67', 'request_id': 'req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.024494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08cb3a4c593a42148a0441a19aff0b67', 'request_id': 'req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.036696 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] The token cache is being invalidate because project 08cb3a4c593a42148a0441a19aff0b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:33:42.036989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.037605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.039117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1271/2554] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:33:41 2022] DELETE /identity/v3/projects/08cb3a4c593a42148a0441a19aff0b67 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:42.043083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c8292-0cf7-403c-96cc-2599b3ace750 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:42.052878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c8292-0cf7-403c-96cc-2599b3ace750 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:42.108666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11f943-540c-41be-8a4c-2e2bd4d71db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:42.109283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f943-540c-41be-8a4c-2e2bd4d71db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:42.109563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f943-540c-41be-8a4c-2e2bd4d71db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:42.109830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11f943-540c-41be-8a4c-2e2bd4d71db7 admin admin] PATH_INFO: `/v3/users/e6ba65f868ad487b81ee63191cc40b70` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:42.110970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11f943-540c-41be-8a4c-2e2bd4d71db7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ba65f868ad487b81ee63191cc40b70)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:42.112873 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11f943-540c-41be-8a4c-2e2bd4d71db7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:42.114305 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:42.159082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2d11f943-540c-41be-8a4c-2e2bd4d71db7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ba65f868ad487b81ee63191cc40b70'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.165824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2d11f943-540c-41be-8a4c-2e2bd4d71db7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ba65f868ad487b81ee63191cc40b70'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.166599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2d11f943-540c-41be-8a4c-2e2bd4d71db7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ba65f868ad487b81ee63191cc40b70'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.174254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2d11f943-540c-41be-8a4c-2e2bd4d71db7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ba65f868ad487b81ee63191cc40b70'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.186479 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2d11f943-540c-41be-8a4c-2e2bd4d71db7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ba65f868ad487b81ee63191cc40b70'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.192657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1284/2555] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:33:42 2022] DELETE /identity/v3/users/e6ba65f868ad487b81ee63191cc40b70 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:42.470988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:42.472449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:42.497936 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31f15dd-f763-4f36-ae0d-17d9771bd9a6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:42.567289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c10e7-b9a8-4120-b518-c60c350e2ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:42.567944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c10e7-b9a8-4120-b518-c60c350e2ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:42.568067 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c10e7-b9a8-4120-b518-c60c350e2ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:42.568338 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c10e7-b9a8-4120-b518-c60c350e2ec9 admin admin] PATH_INFO: `/v3/projects/c4aab6a8b4464312bd1e21a9b647e5fe` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:42.569431 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c10e7-b9a8-4120-b518-c60c350e2ec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4aab6a8b4464312bd1e21a9b647e5fe)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:42.571351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c10e7-b9a8-4120-b518-c60c350e2ec9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:42.572748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:42.620319 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-922c10e7-b9a8-4120-b518-c60c350e2ec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4aab6a8b4464312bd1e21a9b647e5fe', 'request_id': 'req-922c10e7-b9a8-4120-b518-c60c350e2ec9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.629869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-922c10e7-b9a8-4120-b518-c60c350e2ec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4aab6a8b4464312bd1e21a9b647e5fe', 'request_id': 'req-922c10e7-b9a8-4120-b518-c60c350e2ec9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.636167 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-922c10e7-b9a8-4120-b518-c60c350e2ec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4aab6a8b4464312bd1e21a9b647e5fe', 'request_id': 'req-922c10e7-b9a8-4120-b518-c60c350e2ec9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.648976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-922c10e7-b9a8-4120-b518-c60c350e2ec9 admin admin] The token cache is being invalidate because project c4aab6a8b4464312bd1e21a9b647e5fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:33:42.649264 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-922c10e7-b9a8-4120-b518-c60c350e2ec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.649886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-922c10e7-b9a8-4120-b518-c60c350e2ec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:42.651426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1272/2556] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:33:42 2022] DELETE /identity/v3/projects/c4aab6a8b4464312bd1e21a9b647e5fe => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:42.660322 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6a805-65bf-4688-bdac-9019b7857c05 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:42.660796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6a805-65bf-4688-bdac-9019b7857c05 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:42.661028 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6a805-65bf-4688-bdac-9019b7857c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:42.680451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f5d6a805-65bf-4688-bdac-9019b7857c05 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:42.681637 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f5d6a805-65bf-4688-bdac-9019b7857c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:42.739911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1285/2557] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:42.747904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2aaeb-43f6-421d-a79c-45bf7355a122 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:42.748212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2aaeb-43f6-421d-a79c-45bf7355a122 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:42.748480 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2aaeb-43f6-421d-a79c-45bf7355a122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:42.768668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-47a2aaeb-43f6-421d-a79c-45bf7355a122 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:42.769869 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-47a2aaeb-43f6-421d-a79c-45bf7355a122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:42.777747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1273/2558] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:42.782523 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6a805-65bf-4688-bdac-9019b7857c05 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:42.792165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6a805-65bf-4688-bdac-9019b7857c05 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:42.796738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4fcfd-a90d-498a-bcae-843109e420d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:42.797329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4fcfd-a90d-498a-bcae-843109e420d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:42.797607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4fcfd-a90d-498a-bcae-843109e420d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:42.797879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4fcfd-a90d-498a-bcae-843109e420d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:42.798573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4fcfd-a90d-498a-bcae-843109e420d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:33:42.798858 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4fcfd-a90d-498a-bcae-843109e420d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:42.800444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4fcfd-a90d-498a-bcae-843109e420d5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:42.811842 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1286/2559] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:33:42.816273 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2aaeb-43f6-421d-a79c-45bf7355a122 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:42.826305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2aaeb-43f6-421d-a79c-45bf7355a122 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:42.830914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19415f2d-637f-445d-bcba-a4190b5d4bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:42.831504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19415f2d-637f-445d-bcba-a4190b5d4bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:42.831783 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19415f2d-637f-445d-bcba-a4190b5d4bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:42.832056 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19415f2d-637f-445d-bcba-a4190b5d4bc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:42.832685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19415f2d-637f-445d-bcba-a4190b5d4bc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:42.834836 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19415f2d-637f-445d-bcba-a4190b5d4bc6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:42.868560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1274/2560] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:33:42 2022] POST /identity/v3/projects => generated 361 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:42.872931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4fcfd-a90d-498a-bcae-843109e420d5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:42.882859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4fcfd-a90d-498a-bcae-843109e420d5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:42.939837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a15e73-4df9-4421-8f57-9f57c8d215b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:42.940439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a15e73-4df9-4421-8f57-9f57c8d215b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:42.940719 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a15e73-4df9-4421-8f57-9f57c8d215b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:42.941029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a15e73-4df9-4421-8f57-9f57c8d215b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:42.941696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a15e73-4df9-4421-8f57-9f57c8d215b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:42.943639 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a15e73-4df9-4421-8f57-9f57c8d215b6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:42.954943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:42.968232 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1287/2561] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:33:42 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:33:42.972339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19415f2d-637f-445d-bcba-a4190b5d4bc6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:42.982808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19415f2d-637f-445d-bcba-a4190b5d4bc6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:42.987453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52af2d3-7b2d-4575-bf58-ec4b3823f2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:42.988035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52af2d3-7b2d-4575-bf58-ec4b3823f2ac admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:42.988311 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52af2d3-7b2d-4575-bf58-ec4b3823f2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:42.988586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52af2d3-7b2d-4575-bf58-ec4b3823f2ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:42.989220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52af2d3-7b2d-4575-bf58-ec4b3823f2ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:42.990854 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52af2d3-7b2d-4575-bf58-ec4b3823f2ac admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:43.001099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1275/2562] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:43.004870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a15e73-4df9-4421-8f57-9f57c8d215b6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:43.015272 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a15e73-4df9-4421-8f57-9f57c8d215b6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:43.019841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212e490-a98f-4588-935c-04fd565234ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:43.020422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212e490-a98f-4588-935c-04fd565234ed admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:43.020696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212e490-a98f-4588-935c-04fd565234ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:43.020967 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212e490-a98f-4588-935c-04fd565234ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:43.021600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212e490-a98f-4588-935c-04fd565234ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:43.023199 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212e490-a98f-4588-935c-04fd565234ed admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:43.032819 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1288/2563] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:43.036896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52af2d3-7b2d-4575-bf58-ec4b3823f2ac admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:43.046757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52af2d3-7b2d-4575-bf58-ec4b3823f2ac admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:43.051463 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd9683-2226-485b-98eb-1b38b26b92a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:43.052088 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd9683-2226-485b-98eb-1b38b26b92a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:43.052363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd9683-2226-485b-98eb-1b38b26b92a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:43.052630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd9683-2226-485b-98eb-1b38b26b92a6 admin admin] PATH_INFO: `/v3/projects/634f10d47b254488989d09b0a96321f8/users/4fa7ad09b9ec4c349c042742b94240c1/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:43.067273 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd9683-2226-485b-98eb-1b38b26b92a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=634f10d47b254488989d09b0a96321f8, user_id=4fa7ad09b9ec4c349c042742b94240c1, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:43.069907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd9683-2226-485b-98eb-1b38b26b92a6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:43.083684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1276/2564] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:43 2022] PUT /identity/v3/projects/634f10d47b254488989d09b0a96321f8/users/4fa7ad09b9ec4c349c042742b94240c1/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:43.095043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb00c38-4f35-4bf4-bfce-0b6ebf636778 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:43.095325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb00c38-4f35-4bf4-bfce-0b6ebf636778 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:43.095600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb00c38-4f35-4bf4-bfce-0b6ebf636778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:43.124404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-4bb00c38-4f35-4bf4-bfce-0b6ebf636778 None None] MFA Rules not processed for user `4fa7ad09b9ec4c349c042742b94240c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:43.125607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4bb00c38-4f35-4bf4-bfce-0b6ebf636778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:43.189006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1289/2565] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:43.228647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd9683-2226-485b-98eb-1b38b26b92a6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:43.230488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-23fd9683-2226-485b-98eb-1b38b26b92a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:43.288034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd9683-2226-485b-98eb-1b38b26b92a6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:43.291664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40fe65-cd7d-4ae7-8698-bb31cf8a5739 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:43.292253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40fe65-cd7d-4ae7-8698-bb31cf8a5739 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:43.292524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40fe65-cd7d-4ae7-8698-bb31cf8a5739 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:43.292790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40fe65-cd7d-4ae7-8698-bb31cf8a5739 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:43.300989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40fe65-cd7d-4ae7-8698-bb31cf8a5739 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:43.302561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40fe65-cd7d-4ae7-8698-bb31cf8a5739 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:43.312278 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1277/2566] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:43.328749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb00c38-4f35-4bf4-bfce-0b6ebf636778 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:43.389720 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb00c38-4f35-4bf4-bfce-0b6ebf636778 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:43.394239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:43.394845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:43.395116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:43.395380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] PATH_INFO: `/v3/users/4fa7ad09b9ec4c349c042742b94240c1` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:43.396480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fa7ad09b9ec4c349c042742b94240c1)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:43.398399 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:43.444932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fa7ad09b9ec4c349c042742b94240c1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:43.452217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fa7ad09b9ec4c349c042742b94240c1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:43.452890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fa7ad09b9ec4c349c042742b94240c1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:43.460304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fa7ad09b9ec4c349c042742b94240c1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:43.470458 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:43.472845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fa7ad09b9ec4c349c042742b94240c1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:43.480823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1290/2567] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:33:43 2022] DELETE /identity/v3/users/4fa7ad09b9ec4c349c042742b94240c1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:43.486035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aff98-033b-470a-9e42-e0b08988356f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:43.486344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aff98-033b-470a-9e42-e0b08988356f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:43.486617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1aff98-033b-470a-9e42-e0b08988356f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:43.506360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-cc1aff98-033b-470a-9e42-e0b08988356f None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:43.507582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cc1aff98-033b-470a-9e42-e0b08988356f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:43.530913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:43.569233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1278/2568] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:43.578843 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:43.580249 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:43.636637 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26985a9d-ff1f-485d-ba69-fbe949b5e52e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:43.640577 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4530a7-03b3-4900-8f34-4bc11e3ce68f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:43.641099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4530a7-03b3-4900-8f34-4bc11e3ce68f service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:43.641524 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4530a7-03b3-4900-8f34-4bc11e3ce68f service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:43.641677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4530a7-03b3-4900-8f34-4bc11e3ce68f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:43.650496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4530a7-03b3-4900-8f34-4bc11e3ce68f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:43.651900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4530a7-03b3-4900-8f34-4bc11e3ce68f service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:43.660980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1291/2569] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:43.920518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1aff98-033b-470a-9e42-e0b08988356f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:43.921988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cc1aff98-033b-470a-9e42-e0b08988356f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:43.934132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1aff98-033b-470a-9e42-e0b08988356f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:43.938905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655f413-1df6-4f83-b5ca-125a18580758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:43.939616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655f413-1df6-4f83-b5ca-125a18580758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:43.939910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655f413-1df6-4f83-b5ca-125a18580758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:43.940159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655f413-1df6-4f83-b5ca-125a18580758 admin admin] PATH_INFO: `/v3/projects/634f10d47b254488989d09b0a96321f8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:43.941265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655f413-1df6-4f83-b5ca-125a18580758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=634f10d47b254488989d09b0a96321f8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:43.943212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655f413-1df6-4f83-b5ca-125a18580758 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:43.990173 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b655f413-1df6-4f83-b5ca-125a18580758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '634f10d47b254488989d09b0a96321f8', 'request_id': 'req-b655f413-1df6-4f83-b5ca-125a18580758'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:43.995646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b655f413-1df6-4f83-b5ca-125a18580758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '634f10d47b254488989d09b0a96321f8', 'request_id': 'req-b655f413-1df6-4f83-b5ca-125a18580758'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:44.001788 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b655f413-1df6-4f83-b5ca-125a18580758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '634f10d47b254488989d09b0a96321f8', 'request_id': 'req-b655f413-1df6-4f83-b5ca-125a18580758'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:44.014269 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b655f413-1df6-4f83-b5ca-125a18580758 admin admin] The token cache is being invalidate because project 634f10d47b254488989d09b0a96321f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:33:44.014555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b655f413-1df6-4f83-b5ca-125a18580758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:44.015279 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b655f413-1df6-4f83-b5ca-125a18580758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:44.016771 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1279/2570] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:33:43 2022] DELETE /identity/v3/projects/634f10d47b254488989d09b0a96321f8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:44.021719 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e5ae6-1d37-4684-8686-7df867714df8 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.021994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e5ae6-1d37-4684-8686-7df867714df8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.022279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e5ae6-1d37-4684-8686-7df867714df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.041520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-071e5ae6-1d37-4684-8686-7df867714df8 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:44.042718 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-071e5ae6-1d37-4684-8686-7df867714df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:44.098883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1292/2571] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:44.107757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24031612-a45a-4ff1-890c-c17122cd2ead None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.108049 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24031612-a45a-4ff1-890c-c17122cd2ead None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.108320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24031612-a45a-4ff1-890c-c17122cd2ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.127646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-24031612-a45a-4ff1-890c-c17122cd2ead None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:44.128850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-24031612-a45a-4ff1-890c-c17122cd2ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:44.136686 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1280/2572] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:44.141319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e5ae6-1d37-4684-8686-7df867714df8 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:44.150702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e5ae6-1d37-4684-8686-7df867714df8 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:44.155830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38995635-931d-477d-b919-0bfd8e75c4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:44.156417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38995635-931d-477d-b919-0bfd8e75c4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.156692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38995635-931d-477d-b919-0bfd8e75c4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.156962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38995635-931d-477d-b919-0bfd8e75c4d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.157602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38995635-931d-477d-b919-0bfd8e75c4d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:33:44.157912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38995635-931d-477d-b919-0bfd8e75c4d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:44.159515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38995635-931d-477d-b919-0bfd8e75c4d1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:44.170864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1293/2573] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:33:44.174896 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24031612-a45a-4ff1-890c-c17122cd2ead None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:44.184130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24031612-a45a-4ff1-890c-c17122cd2ead None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:44.188764 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265323a-e994-4966-832a-585fc6928a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:44.189347 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265323a-e994-4966-832a-585fc6928a54 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.189624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265323a-e994-4966-832a-585fc6928a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.189892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265323a-e994-4966-832a-585fc6928a54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.190542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265323a-e994-4966-832a-585fc6928a54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:44.192450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265323a-e994-4966-832a-585fc6928a54 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:44.220299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1281/2574] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:33:44 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:44.224742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38995635-931d-477d-b919-0bfd8e75c4d1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:44.234939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38995635-931d-477d-b919-0bfd8e75c4d1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:44.288520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7d892-d725-49ba-8bf5-422d3e200b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:44.289111 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d892-d725-49ba-8bf5-422d3e200b97 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.289389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d892-d725-49ba-8bf5-422d3e200b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.289657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7d892-d725-49ba-8bf5-422d3e200b97 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.290390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7d892-d725-49ba-8bf5-422d3e200b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:44.292293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7d892-d725-49ba-8bf5-422d3e200b97 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:44.314558 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1294/2575] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:33:44 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:44.319343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265323a-e994-4966-832a-585fc6928a54 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:44.329674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265323a-e994-4966-832a-585fc6928a54 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:44.334423 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bfc0c-c4d0-4ec7-9730-48dc4443d7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:44.335010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bfc0c-c4d0-4ec7-9730-48dc4443d7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.335284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bfc0c-c4d0-4ec7-9730-48dc4443d7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.335551 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bfc0c-c4d0-4ec7-9730-48dc4443d7d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.336180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bfc0c-c4d0-4ec7-9730-48dc4443d7d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:44.337754 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bfc0c-c4d0-4ec7-9730-48dc4443d7d2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:44.347925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1282/2576] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:44.351898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7d892-d725-49ba-8bf5-422d3e200b97 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:44.362509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7d892-d725-49ba-8bf5-422d3e200b97 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:44.367038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2824c-f866-4f9f-a995-08a291088ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:44.367616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2824c-f866-4f9f-a995-08a291088ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.367894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2824c-f866-4f9f-a995-08a291088ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.368165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2824c-f866-4f9f-a995-08a291088ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.368801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2824c-f866-4f9f-a995-08a291088ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:44.370420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2824c-f866-4f9f-a995-08a291088ee0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:44.381235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1295/2577] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:44.385096 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bfc0c-c4d0-4ec7-9730-48dc4443d7d2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:44.395018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bfc0c-c4d0-4ec7-9730-48dc4443d7d2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:44.399578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f288e74-5f0b-4778-957a-8ea8d1d43a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:44.400208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f288e74-5f0b-4778-957a-8ea8d1d43a5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.400486 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f288e74-5f0b-4778-957a-8ea8d1d43a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.400759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f288e74-5f0b-4778-957a-8ea8d1d43a5c admin admin] PATH_INFO: `/v3/projects/63fcc61139f64cbd9dfc2b2814c6f2f6/users/09b5f3ee53e6428a864076387c80d0f4/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.414858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f288e74-5f0b-4778-957a-8ea8d1d43a5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=63fcc61139f64cbd9dfc2b2814c6f2f6, user_id=09b5f3ee53e6428a864076387c80d0f4, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:44.417491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f288e74-5f0b-4778-957a-8ea8d1d43a5c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:44.432105 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1283/2578] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:44 2022] PUT /identity/v3/projects/63fcc61139f64cbd9dfc2b2814c6f2f6/users/09b5f3ee53e6428a864076387c80d0f4/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:44.442775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b9885-ac77-41ad-b32d-3b1f35ced601 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.443054 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b9885-ac77-41ad-b32d-3b1f35ced601 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.443320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b9885-ac77-41ad-b32d-3b1f35ced601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.450386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:44.473895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-359b9885-ac77-41ad-b32d-3b1f35ced601 None None] MFA Rules not processed for user `09b5f3ee53e6428a864076387c80d0f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:44.475128 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-359b9885-ac77-41ad-b32d-3b1f35ced601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:44.558499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1296/2579] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:44.562982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f288e74-5f0b-4778-957a-8ea8d1d43a5c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:44.611853 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:44.626955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f288e74-5f0b-4778-957a-8ea8d1d43a5c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:44.631647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186a77a-f9dd-4efe-97bb-f81e6a905a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:44.632256 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186a77a-f9dd-4efe-97bb-f81e6a905a3b admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.632525 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186a77a-f9dd-4efe-97bb-f81e6a905a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.632799 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186a77a-f9dd-4efe-97bb-f81e6a905a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.633437 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186a77a-f9dd-4efe-97bb-f81e6a905a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:44.635376 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186a77a-f9dd-4efe-97bb-f81e6a905a3b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:44.662029 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1284/2580] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:33:44 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:44.666223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b9885-ac77-41ad-b32d-3b1f35ced601 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:44.677270 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b9885-ac77-41ad-b32d-3b1f35ced601 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:44.732208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcec18-0013-4a8f-a06f-9136a8b170a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:44.733139 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcec18-0013-4a8f-a06f-9136a8b170a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.733470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcec18-0013-4a8f-a06f-9136a8b170a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.733767 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcec18-0013-4a8f-a06f-9136a8b170a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.734434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcec18-0013-4a8f-a06f-9136a8b170a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:44.736344 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bcec18-0013-4a8f-a06f-9136a8b170a4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:44.756487 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1297/2581] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:33:44 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:44.760529 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186a77a-f9dd-4efe-97bb-f81e6a905a3b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:44.770598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186a77a-f9dd-4efe-97bb-f81e6a905a3b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:44.775207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce36d4f-e2eb-4072-ac2d-20a7511a88da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:44.775791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce36d4f-e2eb-4072-ac2d-20a7511a88da admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.776074 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce36d4f-e2eb-4072-ac2d-20a7511a88da admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.776346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce36d4f-e2eb-4072-ac2d-20a7511a88da admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.776980 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce36d4f-e2eb-4072-ac2d-20a7511a88da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:44.779103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce36d4f-e2eb-4072-ac2d-20a7511a88da admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:44.789495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1285/2582] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:44.793309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcec18-0013-4a8f-a06f-9136a8b170a4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:44.803788 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bcec18-0013-4a8f-a06f-9136a8b170a4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:44.808914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f67a25-e4a1-4b71-9142-8a79f9493c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:44.809548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f67a25-e4a1-4b71-9142-8a79f9493c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.809861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f67a25-e4a1-4b71-9142-8a79f9493c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.810163 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f67a25-e4a1-4b71-9142-8a79f9493c33 admin admin] PATH_INFO: `/v3/projects/eb0ed1cc2a544e63aacce53116e95030/users/63e349ee5bbc4e06aca36cb0f5fe5439/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.825051 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f67a25-e4a1-4b71-9142-8a79f9493c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0ed1cc2a544e63aacce53116e95030, user_id=63e349ee5bbc4e06aca36cb0f5fe5439, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:44.827944 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f67a25-e4a1-4b71-9142-8a79f9493c33 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:44.846156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1298/2583] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:44 2022] PUT /identity/v3/projects/eb0ed1cc2a544e63aacce53116e95030/users/63e349ee5bbc4e06aca36cb0f5fe5439/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:44.850302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce36d4f-e2eb-4072-ac2d-20a7511a88da admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:44.861199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce36d4f-e2eb-4072-ac2d-20a7511a88da admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:44.916856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b2609-890e-4eee-8116-6ab5a0d5f022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:44.917454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b2609-890e-4eee-8116-6ab5a0d5f022 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.917733 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b2609-890e-4eee-8116-6ab5a0d5f022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.918005 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b2609-890e-4eee-8116-6ab5a0d5f022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.918673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b2609-890e-4eee-8116-6ab5a0d5f022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:44.920255 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b2609-890e-4eee-8116-6ab5a0d5f022 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:44.929796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1286/2584] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:44.934127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f67a25-e4a1-4b71-9142-8a79f9493c33 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:44.944178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f67a25-e4a1-4b71-9142-8a79f9493c33 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:44.948775 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cb8fb-283a-42f9-92ef-60c6e6618dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:44.949415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cb8fb-283a-42f9-92ef-60c6e6618dee admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:44.949690 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cb8fb-283a-42f9-92ef-60c6e6618dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:44.949961 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cb8fb-283a-42f9-92ef-60c6e6618dee admin admin] PATH_INFO: `/v3/projects/eb0ed1cc2a544e63aacce53116e95030/users/63e349ee5bbc4e06aca36cb0f5fe5439/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:44.952453 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cb8fb-283a-42f9-92ef-60c6e6618dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0ed1cc2a544e63aacce53116e95030, user_id=63e349ee5bbc4e06aca36cb0f5fe5439, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:44.955108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cb8fb-283a-42f9-92ef-60c6e6618dee admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:44.965384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1299/2585] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:44 2022] PUT /identity/v3/projects/eb0ed1cc2a544e63aacce53116e95030/users/63e349ee5bbc4e06aca36cb0f5fe5439/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:45.030458 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f870-fce4-497f-b033-d7ac5cab70f3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:45.030651 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f870-fce4-497f-b033-d7ac5cab70f3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:45.030921 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29f870-fce4-497f-b033-d7ac5cab70f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:45.058192 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6c29f870-fce4-497f-b033-d7ac5cab70f3 None None] MFA Rules not processed for user `63e349ee5bbc4e06aca36cb0f5fe5439`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:45.059397 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6c29f870-fce4-497f-b033-d7ac5cab70f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:45.127907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1287/2586] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:45.139133 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cb8fb-283a-42f9-92ef-60c6e6618dee admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:45.140833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-397cb8fb-283a-42f9-92ef-60c6e6618dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:45.190310 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:45.202553 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cb8fb-283a-42f9-92ef-60c6e6618dee admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:45.206208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1658e15-f688-45f9-8c74-a1595554deb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:45.206814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1658e15-f688-45f9-8c74-a1595554deb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:45.207083 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1658e15-f688-45f9-8c74-a1595554deb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:45.207353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1658e15-f688-45f9-8c74-a1595554deb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:45.261799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1658e15-f688-45f9-8c74-a1595554deb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:45.263872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1658e15-f688-45f9-8c74-a1595554deb9 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:45.272418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1300/2587] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:47.811580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29f870-fce4-497f-b033-d7ac5cab70f3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:47.859594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:47.876516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29f870-fce4-497f-b033-d7ac5cab70f3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:47.881003 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:47.881625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:47.881899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:47.882196 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] PATH_INFO: `/v3/users/09b5f3ee53e6428a864076387c80d0f4` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:47.883301 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b5f3ee53e6428a864076387c80d0f4)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:47.885225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:47.929102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b5f3ee53e6428a864076387c80d0f4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:47.929769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b5f3ee53e6428a864076387c80d0f4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:47.936610 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b5f3ee53e6428a864076387c80d0f4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:47.943886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b5f3ee53e6428a864076387c80d0f4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:47.956267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b5f3ee53e6428a864076387c80d0f4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:47.961906 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1288/2588] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:33:47 2022] DELETE /identity/v3/users/09b5f3ee53e6428a864076387c80d0f4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:47.967115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a4c9d-15d9-41f0-b891-b2c89c1e4f03 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:47.967616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a4c9d-15d9-41f0-b891-b2c89c1e4f03 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:47.967723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a4c9d-15d9-41f0-b891-b2c89c1e4f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:47.987344 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6c6a4c9d-15d9-41f0-b891-b2c89c1e4f03 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:47.988517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6c6a4c9d-15d9-41f0-b891-b2c89c1e4f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:48.045635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1301/2589] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:48.055237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:48.056676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:48.113631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d8a08-12e0-4b4a-8921-3499fa1c306d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:48.117456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0169f-b9ae-4fd0-be6a-11d94329722d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:48.118051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0169f-b9ae-4fd0-be6a-11d94329722d service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:48.118366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0169f-b9ae-4fd0-be6a-11d94329722d service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:48.118638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b0169f-b9ae-4fd0-be6a-11d94329722d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:48.127501 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0169f-b9ae-4fd0-be6a-11d94329722d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:48.128914 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b0169f-b9ae-4fd0-be6a-11d94329722d service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:48.138858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1289/2590] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:48.395379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a4c9d-15d9-41f0-b891-b2c89c1e4f03 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:48.396853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6c6a4c9d-15d9-41f0-b891-b2c89c1e4f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:48.410723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a4c9d-15d9-41f0-b891-b2c89c1e4f03 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:48.415295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:48.415953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:48.416230 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:48.416501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] PATH_INFO: `/v3/projects/63fcc61139f64cbd9dfc2b2814c6f2f6` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:48.417734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63fcc61139f64cbd9dfc2b2814c6f2f6)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:48.419573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:48.465671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63fcc61139f64cbd9dfc2b2814c6f2f6', 'request_id': 'req-21e3607f-2f0b-44ed-8742-42e120f7c4a4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:48.471027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63fcc61139f64cbd9dfc2b2814c6f2f6', 'request_id': 'req-21e3607f-2f0b-44ed-8742-42e120f7c4a4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:48.476994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63fcc61139f64cbd9dfc2b2814c6f2f6', 'request_id': 'req-21e3607f-2f0b-44ed-8742-42e120f7c4a4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:48.489477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] The token cache is being invalidate because project 63fcc61139f64cbd9dfc2b2814c6f2f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:33:48.489756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:48.490387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:48.491823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1302/2591] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:33:48 2022] DELETE /identity/v3/projects/63fcc61139f64cbd9dfc2b2814c6f2f6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:48.495877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0169f-b9ae-4fd0-be6a-11d94329722d service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:48.505934 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b0169f-b9ae-4fd0-be6a-11d94329722d service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:48.561256 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259745a6-262d-4323-a9c9-0716618fc8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:48.561878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259745a6-262d-4323-a9c9-0716618fc8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:48.562185 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259745a6-262d-4323-a9c9-0716618fc8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:48.562460 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259745a6-262d-4323-a9c9-0716618fc8bd admin admin] PATH_INFO: `/v3/users/63e349ee5bbc4e06aca36cb0f5fe5439` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:48.563579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259745a6-262d-4323-a9c9-0716618fc8bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=63e349ee5bbc4e06aca36cb0f5fe5439)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:48.565486 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259745a6-262d-4323-a9c9-0716618fc8bd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:48.570729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:48.613695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-259745a6-262d-4323-a9c9-0716618fc8bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63e349ee5bbc4e06aca36cb0f5fe5439'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:48.614426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-259745a6-262d-4323-a9c9-0716618fc8bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63e349ee5bbc4e06aca36cb0f5fe5439'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:48.621273 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-259745a6-262d-4323-a9c9-0716618fc8bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63e349ee5bbc4e06aca36cb0f5fe5439'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:48.629893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-259745a6-262d-4323-a9c9-0716618fc8bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63e349ee5bbc4e06aca36cb0f5fe5439'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:48.641663 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-259745a6-262d-4323-a9c9-0716618fc8bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63e349ee5bbc4e06aca36cb0f5fe5439'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:48.647398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1290/2592] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:33:48 2022] DELETE /identity/v3/users/63e349ee5bbc4e06aca36cb0f5fe5439 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:48.928248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:48.929704 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:48.944426 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3607f-2f0b-44ed-8742-42e120f7c4a4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:48.964965 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:49.005752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c478219-370c-4628-9d8a-c7b62f81d479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:49.006451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c478219-370c-4628-9d8a-c7b62f81d479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:49.006756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c478219-370c-4628-9d8a-c7b62f81d479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:49.006990 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c478219-370c-4628-9d8a-c7b62f81d479 admin admin] PATH_INFO: `/v3/projects/eb0ed1cc2a544e63aacce53116e95030` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:49.008153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c478219-370c-4628-9d8a-c7b62f81d479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb0ed1cc2a544e63aacce53116e95030)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:49.010107 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c478219-370c-4628-9d8a-c7b62f81d479 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:49.056598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2c478219-370c-4628-9d8a-c7b62f81d479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb0ed1cc2a544e63aacce53116e95030', 'request_id': 'req-2c478219-370c-4628-9d8a-c7b62f81d479'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:49.062175 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2c478219-370c-4628-9d8a-c7b62f81d479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb0ed1cc2a544e63aacce53116e95030', 'request_id': 'req-2c478219-370c-4628-9d8a-c7b62f81d479'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:49.068255 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2c478219-370c-4628-9d8a-c7b62f81d479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb0ed1cc2a544e63aacce53116e95030', 'request_id': 'req-2c478219-370c-4628-9d8a-c7b62f81d479'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:49.081408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2c478219-370c-4628-9d8a-c7b62f81d479 admin admin] The token cache is being invalidate because project eb0ed1cc2a544e63aacce53116e95030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:33:49.081700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2c478219-370c-4628-9d8a-c7b62f81d479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:49.082371 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2c478219-370c-4628-9d8a-c7b62f81d479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:33:49.083900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1303/2593] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:33:48 2022] DELETE /identity/v3/projects/eb0ed1cc2a544e63aacce53116e95030 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:49.088961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fb8e3-8aee-4fde-8aad-16f940179930 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:49.089457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fb8e3-8aee-4fde-8aad-16f940179930 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:49.089561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fb8e3-8aee-4fde-8aad-16f940179930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:49.111422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b06fb8e3-8aee-4fde-8aad-16f940179930 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:49.112653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b06fb8e3-8aee-4fde-8aad-16f940179930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:49.171209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1291/2594] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:49.178950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193397f-9fe9-4c17-aa93-5bcf7ba017c9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:49.179238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193397f-9fe9-4c17-aa93-5bcf7ba017c9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:49.179564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193397f-9fe9-4c17-aa93-5bcf7ba017c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:49.198943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8193397f-9fe9-4c17-aa93-5bcf7ba017c9 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:49.200146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8193397f-9fe9-4c17-aa93-5bcf7ba017c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:49.207993 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1304/2595] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:49.212236 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fb8e3-8aee-4fde-8aad-16f940179930 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:49.222563 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fb8e3-8aee-4fde-8aad-16f940179930 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:49.227268 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b82f9-884e-49c8-9eee-44e6b3eda428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:49.227847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b82f9-884e-49c8-9eee-44e6b3eda428 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:49.228123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b82f9-884e-49c8-9eee-44e6b3eda428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:49.228391 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b82f9-884e-49c8-9eee-44e6b3eda428 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:49.229029 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b82f9-884e-49c8-9eee-44e6b3eda428 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:33:49.229326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b82f9-884e-49c8-9eee-44e6b3eda428 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:49.230976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304b82f9-884e-49c8-9eee-44e6b3eda428 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:49.243249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1292/2596] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:33:49.247459 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193397f-9fe9-4c17-aa93-5bcf7ba017c9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:49.256536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193397f-9fe9-4c17-aa93-5bcf7ba017c9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:49.261141 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe034dd-e25b-454d-961d-db2f733bd29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:49.261805 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe034dd-e25b-454d-961d-db2f733bd29b admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:49.262095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe034dd-e25b-454d-961d-db2f733bd29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:49.262380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe034dd-e25b-454d-961d-db2f733bd29b admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:49.263003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe034dd-e25b-454d-961d-db2f733bd29b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:49.264924 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe034dd-e25b-454d-961d-db2f733bd29b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:49.291530 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1305/2597] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:33:49 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:33:49.296544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b82f9-884e-49c8-9eee-44e6b3eda428 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:49.306722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b82f9-884e-49c8-9eee-44e6b3eda428 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:49.362451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02cb38-ad28-41f1-96f9-ecea6910f847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:49.363047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02cb38-ad28-41f1-96f9-ecea6910f847 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:49.363325 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02cb38-ad28-41f1-96f9-ecea6910f847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:49.363594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02cb38-ad28-41f1-96f9-ecea6910f847 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:49.364220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02cb38-ad28-41f1-96f9-ecea6910f847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:49.366134 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02cb38-ad28-41f1-96f9-ecea6910f847 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:49.388859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1293/2598] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:33:49 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:33:49.393137 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe034dd-e25b-454d-961d-db2f733bd29b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:49.403319 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe034dd-e25b-454d-961d-db2f733bd29b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:49.408483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a790b6f-6345-40c8-b2a5-98f336c64c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:49.409066 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a790b6f-6345-40c8-b2a5-98f336c64c81 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:49.409345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a790b6f-6345-40c8-b2a5-98f336c64c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:49.409616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a790b6f-6345-40c8-b2a5-98f336c64c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:49.410315 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a790b6f-6345-40c8-b2a5-98f336c64c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:49.411889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a790b6f-6345-40c8-b2a5-98f336c64c81 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:49.421627 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1306/2599] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:49.425317 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02cb38-ad28-41f1-96f9-ecea6910f847 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:49.430326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:49.435873 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02cb38-ad28-41f1-96f9-ecea6910f847 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:49.441613 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ff9b0-b9c1-4c69-806d-92913ffbe511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:49.442396 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ff9b0-b9c1-4c69-806d-92913ffbe511 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:49.442862 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ff9b0-b9c1-4c69-806d-92913ffbe511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:49.443192 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ff9b0-b9c1-4c69-806d-92913ffbe511 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:49.443978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ff9b0-b9c1-4c69-806d-92913ffbe511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:49.445547 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ff9b0-b9c1-4c69-806d-92913ffbe511 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:49.456724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1294/2600] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:33:49.460754 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a790b6f-6345-40c8-b2a5-98f336c64c81 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:49.471385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a790b6f-6345-40c8-b2a5-98f336c64c81 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:49.475982 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87526a2c-bfa9-4634-95a8-3aab3442d030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:49.476624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87526a2c-bfa9-4634-95a8-3aab3442d030 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:49.476898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87526a2c-bfa9-4634-95a8-3aab3442d030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:49.477167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87526a2c-bfa9-4634-95a8-3aab3442d030 admin admin] PATH_INFO: `/v3/projects/d0504ceeb0b94f309ec98c4702ba13f2/users/9ac019171ef44ae7ae097eb8892f82cc/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:49.491621 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87526a2c-bfa9-4634-95a8-3aab3442d030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0504ceeb0b94f309ec98c4702ba13f2, user_id=9ac019171ef44ae7ae097eb8892f82cc, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:49.494289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87526a2c-bfa9-4634-95a8-3aab3442d030 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:49.507012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1307/2601] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:33:49 2022] PUT /identity/v3/projects/d0504ceeb0b94f309ec98c4702ba13f2/users/9ac019171ef44ae7ae097eb8892f82cc/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:33:49.517910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123cadd-a7da-457a-80bb-c8f76deff64a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:49.518228 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123cadd-a7da-457a-80bb-c8f76deff64a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:49.518498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123cadd-a7da-457a-80bb-c8f76deff64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:49.545950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2123cadd-a7da-457a-80bb-c8f76deff64a None None] MFA Rules not processed for user `9ac019171ef44ae7ae097eb8892f82cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:49.547226 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2123cadd-a7da-457a-80bb-c8f76deff64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:49.610287 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1295/2602] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:49.621578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87526a2c-bfa9-4634-95a8-3aab3442d030 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:33:49.623289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-87526a2c-bfa9-4634-95a8-3aab3442d030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:49.674861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:33:49.678983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87526a2c-bfa9-4634-95a8-3aab3442d030 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:33:49.684268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537325d-d600-483a-a77f-d82129106d11 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc89c1a7e684612a73ec9fef6bda780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:33:49.685552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537325d-d600-483a-a77f-d82129106d11 service glance] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:49.686002 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537325d-d600-483a-a77f-d82129106d11 service glance] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:49.686335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537325d-d600-483a-a77f-d82129106d11 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:49.694739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537325d-d600-483a-a77f-d82129106d11 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:33:49.696159 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537325d-d600-483a-a77f-d82129106d11 service glance] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:33:49.705489 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1308/2603] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:33:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:33:52.401527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375277e-dea1-4a8c-b245-f48014ed4970 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:33:52.402005 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375277e-dea1-4a8c-b245-f48014ed4970 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:33:52.402159 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375277e-dea1-4a8c-b245-f48014ed4970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:33:52.421337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6375277e-dea1-4a8c-b245-f48014ed4970 None None] MFA Rules not processed for user `cac1528c35fb464db00b5d18396b8eac`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:33:52.422563 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6375277e-dea1-4a8c-b245-f48014ed4970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:33:52.476762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1296/2604] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:33:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:33:59.700920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:34:02.470235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=61 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:34:04.912836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537325d-d600-483a-a77f-d82129106d11 service glance] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:34:04.914335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e537325d-d600-483a-a77f-d82129106d11 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:04.980856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537325d-d600-483a-a77f-d82129106d11 service glance] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:34:04.984811 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0742d01-f393-4a5e-a037-6bd77cbe9b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:34:04.985536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0742d01-f393-4a5e-a037-6bd77cbe9b8f service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:04.985880 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0742d01-f393-4a5e-a037-6bd77cbe9b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:04.986249 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0742d01-f393-4a5e-a037-6bd77cbe9b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:04.988004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f0742d01-f393-4a5e-a037-6bd77cbe9b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:05.076120 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0742d01-f393-4a5e-a037-6bd77cbe9b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:34:05.077587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0742d01-f393-4a5e-a037-6bd77cbe9b8f service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:34:05.090103 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1309/2605] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:34:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:34:06.263238 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375277e-dea1-4a8c-b245-f48014ed4970 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:34:06.273380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375277e-dea1-4a8c-b245-f48014ed4970 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:34:06.277135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de736b54-f059-4ac7-812a-c3852757ce55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:34:06.277835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de736b54-f059-4ac7-812a-c3852757ce55 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:06.278131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de736b54-f059-4ac7-812a-c3852757ce55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:06.278402 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de736b54-f059-4ac7-812a-c3852757ce55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:06.280109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-de736b54-f059-4ac7-812a-c3852757ce55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:06.297719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de736b54-f059-4ac7-812a-c3852757ce55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:34:06.299179 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de736b54-f059-4ac7-812a-c3852757ce55 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:34:06.310098 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1297/2606] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:34:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:34:15.085237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=59 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:34:16.305202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:34:16.318887 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0742d01-f393-4a5e-a037-6bd77cbe9b8f service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:34:16.320327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f0742d01-f393-4a5e-a037-6bd77cbe9b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:16.333810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0742d01-f393-4a5e-a037-6bd77cbe9b8f service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:34:16.339254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb92bd-5463-4bce-a8fe-b494dd50f8d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:34:16.339841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb92bd-5463-4bce-a8fe-b494dd50f8d0 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:16.340118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb92bd-5463-4bce-a8fe-b494dd50f8d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:16.340387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb92bd-5463-4bce-a8fe-b494dd50f8d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:16.342044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-52eb92bd-5463-4bce-a8fe-b494dd50f8d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:16.420193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb92bd-5463-4bce-a8fe-b494dd50f8d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:34:16.421772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb92bd-5463-4bce-a8fe-b494dd50f8d0 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:34:16.440644 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1310/2607] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:34:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:34:26.437483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=41 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:34:37.044367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de736b54-f059-4ac7-812a-c3852757ce55 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:34:37.104700 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de736b54-f059-4ac7-812a-c3852757ce55 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:34:37.109635 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:34:37.110279 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:37.111179 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:37.111451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] PATH_INFO: `/v3/users/9ac019171ef44ae7ae097eb8892f82cc` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:37.112559 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ac019171ef44ae7ae097eb8892f82cc)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:34:37.114515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:34:37.158785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ac019171ef44ae7ae097eb8892f82cc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:34:37.159425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ac019171ef44ae7ae097eb8892f82cc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:34:37.166264 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ac019171ef44ae7ae097eb8892f82cc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:34:37.172597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ac019171ef44ae7ae097eb8892f82cc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:34:37.184382 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ac019171ef44ae7ae097eb8892f82cc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:34:37.190717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1298/2608] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:34:37 2022] DELETE /identity/v3/users/9ac019171ef44ae7ae097eb8892f82cc => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:34:37.195197 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fa506-39d5-49e6-a5d3-109184af3ecf None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:37.195671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fa506-39d5-49e6-a5d3-109184af3ecf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:37.195774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fa506-39d5-49e6-a5d3-109184af3ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:37.214394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3a5fa506-39d5-49e6-a5d3-109184af3ecf None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:34:37.215569 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3a5fa506-39d5-49e6-a5d3-109184af3ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:37.273034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1311/2609] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:34:37.282757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:34:37.284327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:37.343064 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c3628-bddb-4739-884c-80507f181cb3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:34:37.346755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b323e4-a65d-449b-9ce4-a67e61d72e6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:34:37.347356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b323e4-a65d-449b-9ce4-a67e61d72e6a service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:37.347633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b323e4-a65d-449b-9ce4-a67e61d72e6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:37.347902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b323e4-a65d-449b-9ce4-a67e61d72e6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:37.357152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b323e4-a65d-449b-9ce4-a67e61d72e6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:34:37.358633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b323e4-a65d-449b-9ce4-a67e61d72e6a service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:34:37.369426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1299/2610] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:34:37.641115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fa506-39d5-49e6-a5d3-109184af3ecf None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:34:37.642612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3a5fa506-39d5-49e6-a5d3-109184af3ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:37.654210 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fa506-39d5-49e6-a5d3-109184af3ecf None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:34:37.659948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:34:37.660431 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:37.660752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:37.666041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] PATH_INFO: `/v3/projects/d0504ceeb0b94f309ec98c4702ba13f2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:37.666041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0504ceeb0b94f309ec98c4702ba13f2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:34:37.666041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:34:37.717678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0504ceeb0b94f309ec98c4702ba13f2', 'request_id': 'req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:34:37.725258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0504ceeb0b94f309ec98c4702ba13f2', 'request_id': 'req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:34:37.734039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0504ceeb0b94f309ec98c4702ba13f2', 'request_id': 'req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:34:37.748252 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] The token cache is being invalidate because project d0504ceeb0b94f309ec98c4702ba13f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:34:37.748538 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:34:37.749204 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:34:37.750745 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1312/2611] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:34:37 2022] DELETE /identity/v3/projects/d0504ceeb0b94f309ec98c4702ba13f2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:34:39.274647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d81a7-0c44-40f1-82f2-01484a95d631 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:39.275123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d81a7-0c44-40f1-82f2-01484a95d631 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:39.275232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d81a7-0c44-40f1-82f2-01484a95d631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:39.308165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:34:39.328436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a94d81a7-0c44-40f1-82f2-01484a95d631 None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:34:39.329653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a94d81a7-0c44-40f1-82f2-01484a95d631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:39.396875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1300/2612] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:34:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:34:39.448694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:34:39.450210 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:39.539589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cd0ce-ceaf-4dea-aac0-5d988a87d0d1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:34:39.544665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e0568-584c-4ab4-8b41-659595a26168 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:34:39.545286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e0568-584c-4ab4-8b41-659595a26168 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:39.545562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e0568-584c-4ab4-8b41-659595a26168 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:39.545829 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e0568-584c-4ab4-8b41-659595a26168 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:39.551907 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:34:39.557363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e0568-584c-4ab4-8b41-659595a26168 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:34:39.559256 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e0568-584c-4ab4-8b41-659595a26168 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:34:39.570618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1313/2613] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:34:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:34:49.394005 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=30 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:34:49.433109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d81a7-0c44-40f1-82f2-01484a95d631 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:34:49.434675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a94d81a7-0c44-40f1-82f2-01484a95d631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:49.490024 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d81a7-0c44-40f1-82f2-01484a95d631 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:34:49.493701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b21ad7c-bd22-47ba-bcd2-ca3c2847b2e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:34:49.494282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21ad7c-bd22-47ba-bcd2-ca3c2847b2e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:49.494560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21ad7c-bd22-47ba-bcd2-ca3c2847b2e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:49.494831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b21ad7c-bd22-47ba-bcd2-ca3c2847b2e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:49.496566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4b21ad7c-bd22-47ba-bcd2-ca3c2847b2e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:49.505895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b21ad7c-bd22-47ba-bcd2-ca3c2847b2e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:34:49.507345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b21ad7c-bd22-47ba-bcd2-ca3c2847b2e3 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:34:49.515224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1301/2614] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:34:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:34:49.567702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:34:53.129486 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59178baf-2dbf-4611-9093-595ac7acb059 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:53.130189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59178baf-2dbf-4611-9093-595ac7acb059 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:53.130189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59178baf-2dbf-4611-9093-595ac7acb059 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:53.131642 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1314/2615] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:34:53.137704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967eb13-694d-4be4-9be7-5d9e19190909 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:53.138242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967eb13-694d-4be4-9be7-5d9e19190909 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:53.138364 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967eb13-694d-4be4-9be7-5d9e19190909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:53.180861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a967eb13-694d-4be4-9be7-5d9e19190909 None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:34:53.182109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a967eb13-694d-4be4-9be7-5d9e19190909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:53.258528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1302/2616] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:34:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:34:53.271038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59178baf-2dbf-4611-9093-595ac7acb059 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:34:53.272709 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-59178baf-2dbf-4611-9093-595ac7acb059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:53.352363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59178baf-2dbf-4611-9093-595ac7acb059 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:34:53.355905 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead441b-7a5f-4cda-9dd5-b8f4a35440fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6cc89c1a7e684612a73ec9fef6bda780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:34:53.356751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead441b-7a5f-4cda-9dd5-b8f4a35440fc service glance] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:53.357033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead441b-7a5f-4cda-9dd5-b8f4a35440fc service glance] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:53.357298 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead441b-7a5f-4cda-9dd5-b8f4a35440fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:53.367282 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead441b-7a5f-4cda-9dd5-b8f4a35440fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:34:53.368698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead441b-7a5f-4cda-9dd5-b8f4a35440fc service glance] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:34:53.379671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1315/2617] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:34:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:34:53.449834 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cabb8-38af-4937-b87b-d33b22f01294 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:53.450324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cabb8-38af-4937-b87b-d33b22f01294 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:53.450652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cabb8-38af-4937-b87b-d33b22f01294 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:53.452242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1303/2618] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:34:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:34:53.459145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af0acd-152a-4f08-a169-bb43cca6cbef None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:53.459444 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af0acd-152a-4f08-a169-bb43cca6cbef None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:53.461358 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af0acd-152a-4f08-a169-bb43cca6cbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:53.496719 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b7af0acd-152a-4f08-a169-bb43cca6cbef None None] MFA Rules not processed for user `5d906452199844149966c9c86db4684e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:34:53.497841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b7af0acd-152a-4f08-a169-bb43cca6cbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:53.505315 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1316/2619] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:34:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:34:53.517087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cabb8-38af-4937-b87b-d33b22f01294 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:34:53.528727 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cabb8-38af-4937-b87b-d33b22f01294 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:34:53.532259 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dd914-4fb9-4977-9957-52efecc31340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:34:53.532875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dd914-4fb9-4977-9957-52efecc31340 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:53.533153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dd914-4fb9-4977-9957-52efecc31340 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:53.533427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dd914-4fb9-4977-9957-52efecc31340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:53.540308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88dd914-4fb9-4977-9957-52efecc31340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:34:53.541725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88dd914-4fb9-4977-9957-52efecc31340 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:34:53.549330 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1304/2620] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:34:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:34:53.631258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a45cb-24eb-44db-8557-b65da2866095 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:53.631541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a45cb-24eb-44db-8557-b65da2866095 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:53.631813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a45cb-24eb-44db-8557-b65da2866095 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:53.633157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1317/2621] 10.222.0.55 () {54 vars in 921 bytes} [Tue Sep 27 17:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 27 17:34:53.637971 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221c090-151b-4151-852c-a6ab01589582 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:53.638282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221c090-151b-4151-852c-a6ab01589582 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:53.638558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221c090-151b-4151-852c-a6ab01589582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:53.658401 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c221c090-151b-4151-852c-a6ab01589582 None None] MFA Rules not processed for user `5537185f42474cfd86fca70fa8beddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:34:53.659590 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c221c090-151b-4151-852c-a6ab01589582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:53.668743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1305/2622] 10.222.0.55 () {58 vars in 1018 bytes} [Tue Sep 27 17:34:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:34:53.679833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a45cb-24eb-44db-8557-b65da2866095 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:34:53.681502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-906a45cb-24eb-44db-8557-b65da2866095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:53.694673 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a45cb-24eb-44db-8557-b65da2866095 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:34:53.698342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387cdbd6-14b1-4e2f-b69d-920fc8891d86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:34:53.698922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387cdbd6-14b1-4e2f-b69d-920fc8891d86 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:53.699198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387cdbd6-14b1-4e2f-b69d-920fc8891d86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:53.699465 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387cdbd6-14b1-4e2f-b69d-920fc8891d86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:53.709868 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387cdbd6-14b1-4e2f-b69d-920fc8891d86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:34:53.711327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387cdbd6-14b1-4e2f-b69d-920fc8891d86 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:34:53.724486 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1318/2623] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:34:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:34:55.467850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a828d34-5ab6-4ddc-a089-3a45ceb71428 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:55.468342 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a828d34-5ab6-4ddc-a089-3a45ceb71428 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:55.468463 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a828d34-5ab6-4ddc-a089-3a45ceb71428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:55.488546 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2a828d34-5ab6-4ddc-a089-3a45ceb71428 None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:34:55.489737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2a828d34-5ab6-4ddc-a089-3a45ceb71428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:34:55.496910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1306/2624] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:34:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:34:55.509396 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387cdbd6-14b1-4e2f-b69d-920fc8891d86 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:34:55.521124 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387cdbd6-14b1-4e2f-b69d-920fc8891d86 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:34:55.526256 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7c4de-c263-452d-b226-dd3e73223298 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:34:55.526845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7c4de-c263-452d-b226-dd3e73223298 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:34:55.527115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7c4de-c263-452d-b226-dd3e73223298 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:34:55.527375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7c4de-c263-452d-b226-dd3e73223298 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:34:55.536189 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7c4de-c263-452d-b226-dd3e73223298 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:34:55.537747 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7c4de-c263-452d-b226-dd3e73223298 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:34:55.546017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1319/2625] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:34:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:35:05.483102 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=75 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:35:05.542592 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=56 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:35:13.946573 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a828d34-5ab6-4ddc-a089-3a45ceb71428 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:35:13.955930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a828d34-5ab6-4ddc-a089-3a45ceb71428 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:35:13.959415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab2405-7760-4171-bf94-e5bc82ea19fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:35:13.959996 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab2405-7760-4171-bf94-e5bc82ea19fe service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:35:13.960268 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab2405-7760-4171-bf94-e5bc82ea19fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:35:13.960534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab2405-7760-4171-bf94-e5bc82ea19fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:35:13.962224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8bab2405-7760-4171-bf94-e5bc82ea19fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:14.026209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab2405-7760-4171-bf94-e5bc82ea19fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:35:14.027646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab2405-7760-4171-bf94-e5bc82ea19fe service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:35:14.036621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1307/2626] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:35:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:35:16.415226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd55ede-2aea-4b12-9b6d-54523ff6ac1f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:35:16.415740 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd55ede-2aea-4b12-9b6d-54523ff6ac1f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:35:16.415972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd55ede-2aea-4b12-9b6d-54523ff6ac1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:35:16.436600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5fd55ede-2aea-4b12-9b6d-54523ff6ac1f None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:35:16.438036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5fd55ede-2aea-4b12-9b6d-54523ff6ac1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:16.445760 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1320/2627] 10.222.0.55 () {58 vars in 967 bytes} [Tue Sep 27 17:35:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:35:16.458698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab2405-7760-4171-bf94-e5bc82ea19fe service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:35:16.470778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab2405-7760-4171-bf94-e5bc82ea19fe service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:35:16.476468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6647c-b7d0-40c5-8bb7-5a6a9d42f7da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:35:16.477165 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6647c-b7d0-40c5-8bb7-5a6a9d42f7da service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:35:16.477557 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6647c-b7d0-40c5-8bb7-5a6a9d42f7da service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:35:16.478372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6647c-b7d0-40c5-8bb7-5a6a9d42f7da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:35:16.486658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6647c-b7d0-40c5-8bb7-5a6a9d42f7da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:35:16.488337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6647c-b7d0-40c5-8bb7-5a6a9d42f7da service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:35:16.497235 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1308/2628] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:35:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:35:22.112578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd55ede-2aea-4b12-9b6d-54523ff6ac1f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:35:22.123406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd55ede-2aea-4b12-9b6d-54523ff6ac1f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:35:22.126952 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156b7eb-8b5d-4ad0-ba52-ec9b615ef23c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:35:22.127537 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156b7eb-8b5d-4ad0-ba52-ec9b615ef23c service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:35:22.127811 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156b7eb-8b5d-4ad0-ba52-ec9b615ef23c service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:35:22.128079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156b7eb-8b5d-4ad0-ba52-ec9b615ef23c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:35:22.129763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0156b7eb-8b5d-4ad0-ba52-ec9b615ef23c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:22.191912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156b7eb-8b5d-4ad0-ba52-ec9b615ef23c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:35:22.193933 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0156b7eb-8b5d-4ad0-ba52-ec9b615ef23c service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:35:22.203234 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1321/2629] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:35:26.493460 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=34 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:35:29.395308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6647c-b7d0-40c5-8bb7-5a6a9d42f7da service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:35:29.398396 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7ac6647c-b7d0-40c5-8bb7-5a6a9d42f7da service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:29.413585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6647c-b7d0-40c5-8bb7-5a6a9d42f7da service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:35:29.425866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:35:29.426680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:35:29.427029 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:35:29.427415 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] PATH_INFO: `/v3/users/fa5bf070765946168f7e5937f2b076c1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:35:29.428631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa5bf070765946168f7e5937f2b076c1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:35:29.431815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:35:29.498833 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa5bf070765946168f7e5937f2b076c1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:29.499599 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa5bf070765946168f7e5937f2b076c1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:29.507407 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa5bf070765946168f7e5937f2b076c1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:29.516312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa5bf070765946168f7e5937f2b076c1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:29.529901 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa5bf070765946168f7e5937f2b076c1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:29.536475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1309/2630] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:35:29 2022] DELETE /identity/v3/users/fa5bf070765946168f7e5937f2b076c1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:35:29.710830 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156b7eb-8b5d-4ad0-ba52-ec9b615ef23c service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:35:29.721714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156b7eb-8b5d-4ad0-ba52-ec9b615ef23c service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:35:29.793536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:35:29.794299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:35:29.794697 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:35:29.794970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] PATH_INFO: `/v3/projects/890469995c02465eb9b9ad2d1fb9ab6f` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:35:29.796096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] RBAC: Authorizing `identity:delete_project(project_id=890469995c02465eb9b9ad2d1fb9ab6f)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:35:29.798013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:35:29.845469 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '890469995c02465eb9b9ad2d1fb9ab6f', 'request_id': 'req-eb851820-253c-4efa-9607-1ac2ac54143a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:29.851109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '890469995c02465eb9b9ad2d1fb9ab6f', 'request_id': 'req-eb851820-253c-4efa-9607-1ac2ac54143a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:29.857334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '890469995c02465eb9b9ad2d1fb9ab6f', 'request_id': 'req-eb851820-253c-4efa-9607-1ac2ac54143a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:29.871261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] The token cache is being invalidate because project 890469995c02465eb9b9ad2d1fb9ab6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:35:29.871550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:29.872214 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:29.873696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1322/2631] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:35:29 2022] DELETE /identity/v3/projects/890469995c02465eb9b9ad2d1fb9ab6f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:35:29.877668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:35:29.879113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:29.893106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772a0e8-9ae9-4bff-ad23-dcc20cd58d1b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:35:29.964495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be948d-491a-4600-b4aa-f3527044d2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:35:29.965132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be948d-491a-4600-b4aa-f3527044d2c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:35:29.965416 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be948d-491a-4600-b4aa-f3527044d2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:35:29.965691 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be948d-491a-4600-b4aa-f3527044d2c8 admin admin] PATH_INFO: `/v3/users/264dabfe887e4866a07e9992cc8ba594` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:35:29.966873 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be948d-491a-4600-b4aa-f3527044d2c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=264dabfe887e4866a07e9992cc8ba594)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:35:29.968798 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be948d-491a-4600-b4aa-f3527044d2c8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:35:30.015654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f0be948d-491a-4600-b4aa-f3527044d2c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '264dabfe887e4866a07e9992cc8ba594'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:30.016306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f0be948d-491a-4600-b4aa-f3527044d2c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '264dabfe887e4866a07e9992cc8ba594'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:30.023431 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f0be948d-491a-4600-b4aa-f3527044d2c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '264dabfe887e4866a07e9992cc8ba594'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:30.031498 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f0be948d-491a-4600-b4aa-f3527044d2c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '264dabfe887e4866a07e9992cc8ba594'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:30.040644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:35:30.046700 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f0be948d-491a-4600-b4aa-f3527044d2c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '264dabfe887e4866a07e9992cc8ba594'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:30.054062 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1310/2632] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:35:29 2022] DELETE /identity/v3/users/264dabfe887e4866a07e9992cc8ba594 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:35:30.283714 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:35:30.288124 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:30.345342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb851820-253c-4efa-9607-1ac2ac54143a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:35:30.362146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:35:30.417841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:35:30.418623 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:35:30.419030 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:35:30.419372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] PATH_INFO: `/v3/projects/69f7ac71eb644876b683a468891ea5a7` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:35:30.420748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69f7ac71eb644876b683a468891ea5a7)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:35:30.424675 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:35:30.491254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69f7ac71eb644876b683a468891ea5a7', 'request_id': 'req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:30.499018 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69f7ac71eb644876b683a468891ea5a7', 'request_id': 'req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:30.506305 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69f7ac71eb644876b683a468891ea5a7', 'request_id': 'req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:30.519878 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] The token cache is being invalidate because project 69f7ac71eb644876b683a468891ea5a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:35:30.520170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:30.522180 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:35:30.523751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1323/2633] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:35:30 2022] DELETE /identity/v3/projects/69f7ac71eb644876b683a468891ea5a7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:35:34.785683 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8526f-ccda-4d00-8af1-32ae1ff4abb7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:35:34.786202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8526f-ccda-4d00-8af1-32ae1ff4abb7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:35:34.786304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8526f-ccda-4d00-8af1-32ae1ff4abb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:35:34.806480 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-91a8526f-ccda-4d00-8af1-32ae1ff4abb7 None None] MFA Rules not processed for user `0b02c2ba9a0a48c294a11b42cbfe28e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:35:34.807685 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-91a8526f-ccda-4d00-8af1-32ae1ff4abb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:34.861245 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1311/2634] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:35:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:35:35.178109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:35:35.179696 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:35.235067 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9b6f9-53b7-401a-baad-78b2f17fd9f2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:35:35.238590 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6540e-7aaf-4c06-ab94-82d1c3b53da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:35:35.239165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6540e-7aaf-4c06-ab94-82d1c3b53da9 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:35:35.239439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6540e-7aaf-4c06-ab94-82d1c3b53da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:35:35.239707 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6540e-7aaf-4c06-ab94-82d1c3b53da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:35:35.241386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-79c6540e-7aaf-4c06-ab94-82d1c3b53da9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:35.250494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6540e-7aaf-4c06-ab94-82d1c3b53da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:35:35.251903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6540e-7aaf-4c06-ab94-82d1c3b53da9 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:35:35.260223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1324/2635] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:35:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:35:35.833480 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8526f-ccda-4d00-8af1-32ae1ff4abb7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:35:35.835046 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-91a8526f-ccda-4d00-8af1-32ae1ff4abb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:35.906946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8526f-ccda-4d00-8af1-32ae1ff4abb7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:35:35.911837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c04c7a-6d4c-4fcf-a646-4c28559888b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:35:35.912430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c04c7a-6d4c-4fcf-a646-4c28559888b9 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:35:35.912701 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c04c7a-6d4c-4fcf-a646-4c28559888b9 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:35:35.912967 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c04c7a-6d4c-4fcf-a646-4c28559888b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:35:35.915217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-47c04c7a-6d4c-4fcf-a646-4c28559888b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:35.926231 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c04c7a-6d4c-4fcf-a646-4c28559888b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:35:35.927772 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c04c7a-6d4c-4fcf-a646-4c28559888b9 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:35:35.939071 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1312/2636] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:35:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:35:38.614151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6540e-7aaf-4c06-ab94-82d1c3b53da9 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:35:38.616418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-79c6540e-7aaf-4c06-ab94-82d1c3b53da9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:38.670204 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:35:38.696013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6540e-7aaf-4c06-ab94-82d1c3b53da9 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:35:38.701516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70a243-5d49-4744-923f-022f3f4d4669 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:35:38.702151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70a243-5d49-4744-923f-022f3f4d4669 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:35:38.702430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70a243-5d49-4744-923f-022f3f4d4669 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:35:38.702699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70a243-5d49-4744-923f-022f3f4d4669 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:35:38.704449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ff70a243-5d49-4744-923f-022f3f4d4669 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:38.717574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70a243-5d49-4744-923f-022f3f4d4669 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:35:38.719202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70a243-5d49-4744-923f-022f3f4d4669 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:35:38.730667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1325/2637] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:35:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:35:45.638376 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c04c7a-6d4c-4fcf-a646-4c28559888b9 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:35:45.639644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-47c04c7a-6d4c-4fcf-a646-4c28559888b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:45.656753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c04c7a-6d4c-4fcf-a646-4c28559888b9 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:35:45.662619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966c9ac-c48d-4352-bdbe-ba721259c49b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:35:45.663883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966c9ac-c48d-4352-bdbe-ba721259c49b service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:35:45.664359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966c9ac-c48d-4352-bdbe-ba721259c49b service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:35:45.664837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966c9ac-c48d-4352-bdbe-ba721259c49b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:35:45.667700 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4966c9ac-c48d-4352-bdbe-ba721259c49b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:35:45.683492 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966c9ac-c48d-4352-bdbe-ba721259c49b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:35:45.686022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4966c9ac-c48d-4352-bdbe-ba721259c49b service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:35:45.697149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1313/2638] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:35:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:35:48.725388 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=11 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:35:55.692450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=66,keystone:DELETE=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:36:21.967348 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70a243-5d49-4744-923f-022f3f4d4669 service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:36:21.976529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70a243-5d49-4744-923f-022f3f4d4669 service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:36:21.980666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36770e-5742-45c4-b23d-22263fb7dd0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:36:21.981244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36770e-5742-45c4-b23d-22263fb7dd0c service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:21.981519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36770e-5742-45c4-b23d-22263fb7dd0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:21.981781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36770e-5742-45c4-b23d-22263fb7dd0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:21.983483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0c36770e-5742-45c4-b23d-22263fb7dd0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:36:22.046167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36770e-5742-45c4-b23d-22263fb7dd0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:36:22.047597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c36770e-5742-45c4-b23d-22263fb7dd0c service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:36:22.057056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1326/2639] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:36:25.547662 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966c9ac-c48d-4352-bdbe-ba721259c49b service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:36:25.549190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4966c9ac-c48d-4352-bdbe-ba721259c49b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:36:25.563108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966c9ac-c48d-4352-bdbe-ba721259c49b service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:36:25.567637 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b8b5c7-f6fa-465f-b8e5-84cae7802e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:36:25.568259 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8b5c7-f6fa-465f-b8e5-84cae7802e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:25.568538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8b5c7-f6fa-465f-b8e5-84cae7802e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:25.568808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b8b5c7-f6fa-465f-b8e5-84cae7802e76 admin admin] PATH_INFO: `/v3/users/1dde550901f04ea695b85db4e41d9628` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:25.569931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8b5c7-f6fa-465f-b8e5-84cae7802e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dde550901f04ea695b85db4e41d9628)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:36:25.571882 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b8b5c7-f6fa-465f-b8e5-84cae7802e76 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:36:25.617283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-74b8b5c7-f6fa-465f-b8e5-84cae7802e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dde550901f04ea695b85db4e41d9628'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:36:25.617919 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-74b8b5c7-f6fa-465f-b8e5-84cae7802e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dde550901f04ea695b85db4e41d9628'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:36:25.625198 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-74b8b5c7-f6fa-465f-b8e5-84cae7802e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dde550901f04ea695b85db4e41d9628'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:36:25.631941 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-74b8b5c7-f6fa-465f-b8e5-84cae7802e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dde550901f04ea695b85db4e41d9628'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:36:25.644374 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-74b8b5c7-f6fa-465f-b8e5-84cae7802e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dde550901f04ea695b85db4e41d9628'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:36:25.650160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1314/2640] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:36:25 2022] DELETE /identity/v3/users/1dde550901f04ea695b85db4e41d9628 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:36:25.835823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36770e-5742-45c4-b23d-22263fb7dd0c service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:36:25.847112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36770e-5742-45c4-b23d-22263fb7dd0c service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:36:25.904418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cabcf-7f2e-4674-834b-002a37928fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:36:25.905041 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cabcf-7f2e-4674-834b-002a37928fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:25.905316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cabcf-7f2e-4674-834b-002a37928fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:25.905585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cabcf-7f2e-4674-834b-002a37928fd5 admin admin] PATH_INFO: `/v3/projects/37e99b06d94d4893963a88cd5333ec5b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:25.906825 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cabcf-7f2e-4674-834b-002a37928fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37e99b06d94d4893963a88cd5333ec5b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:36:25.908744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cabcf-7f2e-4674-834b-002a37928fd5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:36:25.956957 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-bf4cabcf-7f2e-4674-834b-002a37928fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37e99b06d94d4893963a88cd5333ec5b', 'request_id': 'req-bf4cabcf-7f2e-4674-834b-002a37928fd5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:36:25.962404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-bf4cabcf-7f2e-4674-834b-002a37928fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37e99b06d94d4893963a88cd5333ec5b', 'request_id': 'req-bf4cabcf-7f2e-4674-834b-002a37928fd5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:36:25.968524 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-bf4cabcf-7f2e-4674-834b-002a37928fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37e99b06d94d4893963a88cd5333ec5b', 'request_id': 'req-bf4cabcf-7f2e-4674-834b-002a37928fd5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:36:25.980606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-bf4cabcf-7f2e-4674-834b-002a37928fd5 admin admin] The token cache is being invalidate because project 37e99b06d94d4893963a88cd5333ec5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:36:25.980887 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-bf4cabcf-7f2e-4674-834b-002a37928fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:36:25.981501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-bf4cabcf-7f2e-4674-834b-002a37928fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:36:25.983002 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1327/2641] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:36:25 2022] DELETE /identity/v3/projects/37e99b06d94d4893963a88cd5333ec5b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:36:25.988101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c520f-a4c9-467a-afa2-a1de7ac7dba7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:25.988408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c520f-a4c9-467a-afa2-a1de7ac7dba7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:25.988679 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c520f-a4c9-467a-afa2-a1de7ac7dba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:26.008987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-212c520f-a4c9-467a-afa2-a1de7ac7dba7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:36:26.010222 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-212c520f-a4c9-467a-afa2-a1de7ac7dba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:36:26.077132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1315/2642] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:36:26.085122 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c36ac9-fcd2-4602-a460-52dc2958f2db None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:26.085343 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c36ac9-fcd2-4602-a460-52dc2958f2db None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:26.085617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c36ac9-fcd2-4602-a460-52dc2958f2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:26.105573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-91c36ac9-fcd2-4602-a460-52dc2958f2db None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:36:26.106799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-91c36ac9-fcd2-4602-a460-52dc2958f2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:36:26.114964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1328/2643] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:36:26.120379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c520f-a4c9-467a-afa2-a1de7ac7dba7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:36:26.129966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c520f-a4c9-467a-afa2-a1de7ac7dba7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:36:26.134541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a2486-c496-4b3c-8a00-16d9b439337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:36:26.135406 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a2486-c496-4b3c-8a00-16d9b439337a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:26.135680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a2486-c496-4b3c-8a00-16d9b439337a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:26.135946 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a2486-c496-4b3c-8a00-16d9b439337a admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:26.136582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a2486-c496-4b3c-8a00-16d9b439337a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:36:26.136876 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a2486-c496-4b3c-8a00-16d9b439337a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:36:26.138466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a2486-c496-4b3c-8a00-16d9b439337a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:36:26.149656 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1316/2644] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:36:26.154583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c36ac9-fcd2-4602-a460-52dc2958f2db None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:36:26.163556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c36ac9-fcd2-4602-a460-52dc2958f2db None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:36:26.168774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaebce-2b56-4d49-b1aa-0d6d8a3a3259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:36:26.169355 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaebce-2b56-4d49-b1aa-0d6d8a3a3259 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:26.169630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaebce-2b56-4d49-b1aa-0d6d8a3a3259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:26.169898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eaebce-2b56-4d49-b1aa-0d6d8a3a3259 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:26.170566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eaebce-2b56-4d49-b1aa-0d6d8a3a3259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:36:26.172445 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eaebce-2b56-4d49-b1aa-0d6d8a3a3259 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:36:26.187200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:36:26.208093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1329/2645] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:36:26 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:36:26.212180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a2486-c496-4b3c-8a00-16d9b439337a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:36:26.222528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a2486-c496-4b3c-8a00-16d9b439337a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:36:26.276767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83246f3-137f-4c41-851b-bbab99392a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:36:26.277352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83246f3-137f-4c41-851b-bbab99392a10 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:26.277625 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83246f3-137f-4c41-851b-bbab99392a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:26.277895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83246f3-137f-4c41-851b-bbab99392a10 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:26.278545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83246f3-137f-4c41-851b-bbab99392a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:36:26.280427 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83246f3-137f-4c41-851b-bbab99392a10 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:36:26.288216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:36:26.303397 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1317/2646] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:36:26 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:36:26.307583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaebce-2b56-4d49-b1aa-0d6d8a3a3259 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:36:26.317959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eaebce-2b56-4d49-b1aa-0d6d8a3a3259 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:36:26.322599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342d8f4-5903-44f1-9153-90a0cad08c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:36:26.323186 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342d8f4-5903-44f1-9153-90a0cad08c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:26.323463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342d8f4-5903-44f1-9153-90a0cad08c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:26.323734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342d8f4-5903-44f1-9153-90a0cad08c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:26.324373 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342d8f4-5903-44f1-9153-90a0cad08c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:36:26.325959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342d8f4-5903-44f1-9153-90a0cad08c85 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:36:26.335831 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1330/2647] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:36:26.340232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83246f3-137f-4c41-851b-bbab99392a10 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:36:26.350917 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83246f3-137f-4c41-851b-bbab99392a10 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:36:26.356158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9774ad-a02f-4e53-85a5-a4c52039a7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:36:26.356742 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9774ad-a02f-4e53-85a5-a4c52039a7cd admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:26.357018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9774ad-a02f-4e53-85a5-a4c52039a7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:26.357289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9774ad-a02f-4e53-85a5-a4c52039a7cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:26.357923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9774ad-a02f-4e53-85a5-a4c52039a7cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:36:26.359823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9774ad-a02f-4e53-85a5-a4c52039a7cd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:36:26.370230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1318/2648] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:36:26.374302 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342d8f4-5903-44f1-9153-90a0cad08c85 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:36:26.384162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342d8f4-5903-44f1-9153-90a0cad08c85 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:36:26.388756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2bf83-eb65-4ba0-8461-a5e632858ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:36:26.389389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2bf83-eb65-4ba0-8461-a5e632858ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:26.389777 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2bf83-eb65-4ba0-8461-a5e632858ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:26.390054 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2bf83-eb65-4ba0-8461-a5e632858ba2 admin admin] PATH_INFO: `/v3/projects/0bd65ea59a6d41219ecf99f5a554056a/users/1b888fd409954909874b5443cfed2838/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:26.405963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2bf83-eb65-4ba0-8461-a5e632858ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd65ea59a6d41219ecf99f5a554056a, user_id=1b888fd409954909874b5443cfed2838, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:36:26.408620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2bf83-eb65-4ba0-8461-a5e632858ba2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:36:26.421959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1331/2649] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:36:26 2022] PUT /identity/v3/projects/0bd65ea59a6d41219ecf99f5a554056a/users/1b888fd409954909874b5443cfed2838/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:36:26.426907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a9842-3194-4e84-9c3a-394ee2082481 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:26.427187 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a9842-3194-4e84-9c3a-394ee2082481 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:26.427456 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a9842-3194-4e84-9c3a-394ee2082481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:26.447063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-cc7a9842-3194-4e84-9c3a-394ee2082481 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:36:26.448356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cc7a9842-3194-4e84-9c3a-394ee2082481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:36:26.506549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1319/2650] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:36:26.516848 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2bf83-eb65-4ba0-8461-a5e632858ba2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:36:26.518631 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1bf2bf83-eb65-4ba0-8461-a5e632858ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:36:26.573439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2bf83-eb65-4ba0-8461-a5e632858ba2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:36:26.577056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a448aa-51cc-46b9-a48d-fba4af4b39b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:36:26.577645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a448aa-51cc-46b9-a48d-fba4af4b39b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:26.577919 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a448aa-51cc-46b9-a48d-fba4af4b39b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:26.578219 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a448aa-51cc-46b9-a48d-fba4af4b39b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:26.586949 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a448aa-51cc-46b9-a48d-fba4af4b39b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:36:26.588962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a448aa-51cc-46b9-a48d-fba4af4b39b6 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:36:26.598115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1332/2651] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:36:28.733351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a9842-3194-4e84-9c3a-394ee2082481 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:36:28.735284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cc7a9842-3194-4e84-9c3a-394ee2082481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:36:28.833762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a9842-3194-4e84-9c3a-394ee2082481 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:36:28.839773 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4da16-69d7-47a9-85b5-837a9d2bce3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:36:28.841770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4da16-69d7-47a9-85b5-837a9d2bce3f service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:28.842307 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4da16-69d7-47a9-85b5-837a9d2bce3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:28.843224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4da16-69d7-47a9-85b5-837a9d2bce3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:28.846800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7bc4da16-69d7-47a9-85b5-837a9d2bce3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:36:28.919669 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4da16-69d7-47a9-85b5-837a9d2bce3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:36:28.922101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4da16-69d7-47a9-85b5-837a9d2bce3f service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:36:28.936749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1320/2652] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:36:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:36:30.376117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016134e-5c2c-4e45-bebf-ae0ada8cf47c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:30.376650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016134e-5c2c-4e45-bebf-ae0ada8cf47c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:30.376650 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016134e-5c2c-4e45-bebf-ae0ada8cf47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:30.405383 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6016134e-5c2c-4e45-bebf-ae0ada8cf47c None None] MFA Rules not processed for user `1b888fd409954909874b5443cfed2838`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:36:30.406699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6016134e-5c2c-4e45-bebf-ae0ada8cf47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:36:30.482938 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1333/2653] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:36:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:36:30.498489 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4da16-69d7-47a9-85b5-837a9d2bce3f service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:36:30.500071 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7bc4da16-69d7-47a9-85b5-837a9d2bce3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:36:30.536749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:36:30.578156 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4da16-69d7-47a9-85b5-837a9d2bce3f service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:36:30.583720 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadfe88-d552-483a-86df-465ebae74dd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:36:30.584334 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadfe88-d552-483a-86df-465ebae74dd3 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:30.584669 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadfe88-d552-483a-86df-465ebae74dd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:30.584886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbadfe88-d552-483a-86df-465ebae74dd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:30.593602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadfe88-d552-483a-86df-465ebae74dd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:36:30.595246 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbadfe88-d552-483a-86df-465ebae74dd3 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:36:30.605664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1321/2654] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:36:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:36:32.121513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a28a7-bd29-4559-bd32-66e9c7b76723 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:32.122034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a28a7-bd29-4559-bd32-66e9c7b76723 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:32.122548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a28a7-bd29-4559-bd32-66e9c7b76723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:32.144076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a37a28a7-bd29-4559-bd32-66e9c7b76723 None None] MFA Rules not processed for user `1b888fd409954909874b5443cfed2838`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:36:32.145301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a37a28a7-bd29-4559-bd32-66e9c7b76723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:36:32.152790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1334/2655] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:36:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:36:32.475036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadfe88-d552-483a-86df-465ebae74dd3 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:36:32.485698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbadfe88-d552-483a-86df-465ebae74dd3 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:36:32.489440 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770a410-c638-4499-97d6-54a90cfd8242 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:36:32.490017 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770a410-c638-4499-97d6-54a90cfd8242 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:32.490332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770a410-c638-4499-97d6-54a90cfd8242 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:32.490602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770a410-c638-4499-97d6-54a90cfd8242 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:32.492345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c770a410-c638-4499-97d6-54a90cfd8242 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:36:32.502116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770a410-c638-4499-97d6-54a90cfd8242 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:36:32.503540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770a410-c638-4499-97d6-54a90cfd8242 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:36:32.512181 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1322/2656] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:36:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:36:40.141428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d444702-4309-4299-b9cb-9b34877f31f1 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:36:40.141914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d444702-4309-4299-b9cb-9b34877f31f1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:36:40.141914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d444702-4309-4299-b9cb-9b34877f31f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:36:40.160457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1d444702-4309-4299-b9cb-9b34877f31f1 None None] MFA Rules not processed for user `cac1528c35fb464db00b5d18396b8eac`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:36:40.161719 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1d444702-4309-4299-b9cb-9b34877f31f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:36:40.169151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1335/2657] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:36:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:36:42.508306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=25 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:36:50.155576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=84,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:00.862945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770a410-c638-4499-97d6-54a90cfd8242 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:00.872306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770a410-c638-4499-97d6-54a90cfd8242 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:00.885332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c35ae-501c-43ca-85d0-88e92a5231b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:00.885916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c35ae-501c-43ca-85d0-88e92a5231b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:00.886223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c35ae-501c-43ca-85d0-88e92a5231b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:00.886494 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c35ae-501c-43ca-85d0-88e92a5231b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:00.898133 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dc1c35ae-501c-43ca-85d0-88e92a5231b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:00.968442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c35ae-501c-43ca-85d0-88e92a5231b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:00.969871 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c35ae-501c-43ca-85d0-88e92a5231b1 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:00.980014 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1323/2658] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:37:00 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:37:10.975127 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=30 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:21.827250 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d444702-4309-4299-b9cb-9b34877f31f1 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:21.828614 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1d444702-4309-4299-b9cb-9b34877f31f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:21.897685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d444702-4309-4299-b9cb-9b34877f31f1 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:21.902699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1cc10-54b1-4f4a-a30b-66d048e8f2c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:21.903301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1cc10-54b1-4f4a-a30b-66d048e8f2c8 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:21.903582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1cc10-54b1-4f4a-a30b-66d048e8f2c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:21.903855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1cc10-54b1-4f4a-a30b-66d048e8f2c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:21.905584 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5dd1cc10-54b1-4f4a-a30b-66d048e8f2c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:21.916607 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1cc10-54b1-4f4a-a30b-66d048e8f2c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:21.918211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1cc10-54b1-4f4a-a30b-66d048e8f2c8 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:21.928402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1336/2659] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:37:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:37:31.924232 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=32 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:35.343956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c35ae-501c-43ca-85d0-88e92a5231b1 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:35.352642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c35ae-501c-43ca-85d0-88e92a5231b1 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:35.356297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257961e-e19a-4ef5-90de-f4bd11aa2815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:35.356883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257961e-e19a-4ef5-90de-f4bd11aa2815 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:35.357807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257961e-e19a-4ef5-90de-f4bd11aa2815 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:35.358099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257961e-e19a-4ef5-90de-f4bd11aa2815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:35.359841 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8257961e-e19a-4ef5-90de-f4bd11aa2815 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:35.371627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257961e-e19a-4ef5-90de-f4bd11aa2815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:35.373028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257961e-e19a-4ef5-90de-f4bd11aa2815 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:35.382448 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1324/2660] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:37:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:37:38.223480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1cc10-54b1-4f4a-a30b-66d048e8f2c8 service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:38.225667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5dd1cc10-54b1-4f4a-a30b-66d048e8f2c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:38.248069 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1cc10-54b1-4f4a-a30b-66d048e8f2c8 service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:38.253169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22669838-cea5-4ed2-8b96-6af14a57d762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:38.253970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22669838-cea5-4ed2-8b96-6af14a57d762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:38.254261 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22669838-cea5-4ed2-8b96-6af14a57d762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:38.254532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22669838-cea5-4ed2-8b96-6af14a57d762 admin admin] PATH_INFO: `/v3/users/0b02c2ba9a0a48c294a11b42cbfe28e3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:38.268408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22669838-cea5-4ed2-8b96-6af14a57d762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b02c2ba9a0a48c294a11b42cbfe28e3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:38.270361 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22669838-cea5-4ed2-8b96-6af14a57d762 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:38.320753 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-22669838-cea5-4ed2-8b96-6af14a57d762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b02c2ba9a0a48c294a11b42cbfe28e3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:38.327556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-22669838-cea5-4ed2-8b96-6af14a57d762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b02c2ba9a0a48c294a11b42cbfe28e3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:38.328211 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-22669838-cea5-4ed2-8b96-6af14a57d762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b02c2ba9a0a48c294a11b42cbfe28e3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:38.335512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-22669838-cea5-4ed2-8b96-6af14a57d762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b02c2ba9a0a48c294a11b42cbfe28e3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:38.348080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-22669838-cea5-4ed2-8b96-6af14a57d762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b02c2ba9a0a48c294a11b42cbfe28e3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:38.353841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1337/2661] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:37:38 2022] DELETE /identity/v3/users/0b02c2ba9a0a48c294a11b42cbfe28e3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:38.523116 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257961e-e19a-4ef5-90de-f4bd11aa2815 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:38.533651 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257961e-e19a-4ef5-90de-f4bd11aa2815 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:38.590910 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:38.592084 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:38.592369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:38.592637 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364 admin admin] PATH_INFO: `/v3/projects/ef13103d3d8b43e1882d4820c8d9247b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:38.602831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef13103d3d8b43e1882d4820c8d9247b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:38.604748 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:38.649976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef13103d3d8b43e1882d4820c8d9247b', 'request_id': 'req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:38.655927 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef13103d3d8b43e1882d4820c8d9247b', 'request_id': 'req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:38.662409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef13103d3d8b43e1882d4820c8d9247b', 'request_id': 'req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:38.674726 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364 admin admin] The token cache is being invalidate because project ef13103d3d8b43e1882d4820c8d9247b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:37:38.675009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:38.675628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f91f5ce0-ced6-4b86-918e-2d5ee8f6e364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:38.677126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1325/2662] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:37:38 2022] DELETE /identity/v3/projects/ef13103d3d8b43e1882d4820c8d9247b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:38.682097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc204e9-5461-4123-bb29-463adb2502f2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:38.682395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc204e9-5461-4123-bb29-463adb2502f2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:38.682668 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc204e9-5461-4123-bb29-463adb2502f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:38.702546 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2cc204e9-5461-4123-bb29-463adb2502f2 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:38.703679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2cc204e9-5461-4123-bb29-463adb2502f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:38.761559 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1338/2663] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:38.770800 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cd5a7-af7d-4bad-adc1-ff22df21be62 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:38.771087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cd5a7-af7d-4bad-adc1-ff22df21be62 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:38.771359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cd5a7-af7d-4bad-adc1-ff22df21be62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:38.790549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-845cd5a7-af7d-4bad-adc1-ff22df21be62 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:38.791752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-845cd5a7-af7d-4bad-adc1-ff22df21be62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:38.799606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1326/2664] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:38.803576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc204e9-5461-4123-bb29-463adb2502f2 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:38.813521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc204e9-5461-4123-bb29-463adb2502f2 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:38.818028 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab5b90-65d3-436e-a3e5-946e49e39557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:38.818641 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab5b90-65d3-436e-a3e5-946e49e39557 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:38.818920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab5b90-65d3-436e-a3e5-946e49e39557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:38.819191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab5b90-65d3-436e-a3e5-946e49e39557 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:38.819864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab5b90-65d3-436e-a3e5-946e49e39557 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:37:38.820165 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab5b90-65d3-436e-a3e5-946e49e39557 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:38.821746 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ab5b90-65d3-436e-a3e5-946e49e39557 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:38.834227 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1339/2665] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:37:38.838420 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cd5a7-af7d-4bad-adc1-ff22df21be62 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:38.847702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cd5a7-af7d-4bad-adc1-ff22df21be62 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:38.852356 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84261a85-ce04-4a2e-a39a-94753c8fecde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:38.852950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84261a85-ce04-4a2e-a39a-94753c8fecde admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:38.853229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84261a85-ce04-4a2e-a39a-94753c8fecde admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:38.853502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84261a85-ce04-4a2e-a39a-94753c8fecde admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:38.854156 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84261a85-ce04-4a2e-a39a-94753c8fecde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:38.856051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84261a85-ce04-4a2e-a39a-94753c8fecde admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:38.883618 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1327/2666] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:37:38 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:37:38.887591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab5b90-65d3-436e-a3e5-946e49e39557 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:38.897910 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab5b90-65d3-436e-a3e5-946e49e39557 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:38.940930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:38.955201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd757025-16c8-4d2b-a17a-46e1ff0bf75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:38.955744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd757025-16c8-4d2b-a17a-46e1ff0bf75a admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:38.956023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd757025-16c8-4d2b-a17a-46e1ff0bf75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:38.956293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd757025-16c8-4d2b-a17a-46e1ff0bf75a admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:38.956932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd757025-16c8-4d2b-a17a-46e1ff0bf75a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:38.958870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd757025-16c8-4d2b-a17a-46e1ff0bf75a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:38.980308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1340/2667] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:37:38 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:37:38.984531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84261a85-ce04-4a2e-a39a-94753c8fecde admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:38.994955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84261a85-ce04-4a2e-a39a-94753c8fecde admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:39.004141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7414c-e213-4813-a307-af6802bbb26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:39.005612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7414c-e213-4813-a307-af6802bbb26f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:39.005898 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7414c-e213-4813-a307-af6802bbb26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:39.006220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7414c-e213-4813-a307-af6802bbb26f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:39.006894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7414c-e213-4813-a307-af6802bbb26f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:39.008481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7414c-e213-4813-a307-af6802bbb26f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:39.019237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1328/2668] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:37:39.023309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd757025-16c8-4d2b-a17a-46e1ff0bf75a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:39.037916 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd757025-16c8-4d2b-a17a-46e1ff0bf75a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:39.046885 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b787b9-6c71-44b4-8920-3d836c342e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:39.047484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b787b9-6c71-44b4-8920-3d836c342e72 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:39.047763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b787b9-6c71-44b4-8920-3d836c342e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:39.048036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b787b9-6c71-44b4-8920-3d836c342e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:39.048677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b787b9-6c71-44b4-8920-3d836c342e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:39.050306 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b787b9-6c71-44b4-8920-3d836c342e72 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:39.061750 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1341/2669] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:37:39.065853 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7414c-e213-4813-a307-af6802bbb26f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:39.075620 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7414c-e213-4813-a307-af6802bbb26f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:39.080884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e3ea6-18ef-4fc0-acd4-9b995e18b213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:39.081512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e3ea6-18ef-4fc0-acd4-9b995e18b213 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:39.081784 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e3ea6-18ef-4fc0-acd4-9b995e18b213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:39.082049 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600e3ea6-18ef-4fc0-acd4-9b995e18b213 admin admin] PATH_INFO: `/v3/projects/42bf5b446a8a499dbe79c022a4e6f72c/users/c13b697eb49c4ad6aaceb47503db4a2d/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:39.096894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e3ea6-18ef-4fc0-acd4-9b995e18b213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42bf5b446a8a499dbe79c022a4e6f72c, user_id=c13b697eb49c4ad6aaceb47503db4a2d, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:39.100586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600e3ea6-18ef-4fc0-acd4-9b995e18b213 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:39.113883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1329/2670] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:37:39 2022] PUT /identity/v3/projects/42bf5b446a8a499dbe79c022a4e6f72c/users/c13b697eb49c4ad6aaceb47503db4a2d/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:39.124509 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719cf09e-b145-4541-8792-77539c366e3e None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:39.124795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719cf09e-b145-4541-8792-77539c366e3e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:39.125074 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719cf09e-b145-4541-8792-77539c366e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:39.153266 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-719cf09e-b145-4541-8792-77539c366e3e None None] MFA Rules not processed for user `c13b697eb49c4ad6aaceb47503db4a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:39.154499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-719cf09e-b145-4541-8792-77539c366e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:39.216445 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1342/2671] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:39 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:39.220922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e3ea6-18ef-4fc0-acd4-9b995e18b213 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:39.226468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:39.284632 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600e3ea6-18ef-4fc0-acd4-9b995e18b213 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:39.289331 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34cc0d-7959-440b-b3f7-6d10cbba48a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:39.289920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34cc0d-7959-440b-b3f7-6d10cbba48a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:39.290230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34cc0d-7959-440b-b3f7-6d10cbba48a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:39.290504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34cc0d-7959-440b-b3f7-6d10cbba48a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:39.291132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34cc0d-7959-440b-b3f7-6d10cbba48a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:39.293031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34cc0d-7959-440b-b3f7-6d10cbba48a9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:39.324493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1330/2672] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:37:39 2022] POST /identity/v3/projects => generated 403 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:37:39.328616 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719cf09e-b145-4541-8792-77539c366e3e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:39.339344 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719cf09e-b145-4541-8792-77539c366e3e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:39.399367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e23ee3-f2f3-40cc-86bc-525fae79bb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:39.399978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e23ee3-f2f3-40cc-86bc-525fae79bb71 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:39.400284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e23ee3-f2f3-40cc-86bc-525fae79bb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:39.400552 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e23ee3-f2f3-40cc-86bc-525fae79bb71 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:39.401241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e23ee3-f2f3-40cc-86bc-525fae79bb71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:39.403213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e23ee3-f2f3-40cc-86bc-525fae79bb71 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:39.427488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1343/2673] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:37:39 2022] POST /identity/v3/users => generated 308 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:37:39.428903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34cc0d-7959-440b-b3f7-6d10cbba48a9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:39.440201 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34cc0d-7959-440b-b3f7-6d10cbba48a9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:39.444942 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef976f2-43fa-429e-8ae8-636905a0db7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:39.445538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef976f2-43fa-429e-8ae8-636905a0db7a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:39.445878 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef976f2-43fa-429e-8ae8-636905a0db7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:39.446199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef976f2-43fa-429e-8ae8-636905a0db7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:39.446796 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef976f2-43fa-429e-8ae8-636905a0db7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:39.448290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef976f2-43fa-429e-8ae8-636905a0db7a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:39.578985 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1331/2674] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:37:39.583096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e23ee3-f2f3-40cc-86bc-525fae79bb71 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:39.593577 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e23ee3-f2f3-40cc-86bc-525fae79bb71 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:39.598036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fa867-9d98-4da7-b2bd-55fb676b72d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:39.598699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fa867-9d98-4da7-b2bd-55fb676b72d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:39.598971 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fa867-9d98-4da7-b2bd-55fb676b72d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:39.599237 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526fa867-9d98-4da7-b2bd-55fb676b72d4 admin admin] PATH_INFO: `/v3/projects/318d6977a377417faf36dade126e62e3/users/c37e78bd9f704128baf701eba05dca20/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:39.614502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526fa867-9d98-4da7-b2bd-55fb676b72d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=318d6977a377417faf36dade126e62e3, user_id=c37e78bd9f704128baf701eba05dca20, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:39.617122 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526fa867-9d98-4da7-b2bd-55fb676b72d4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:39.630446 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1344/2675] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:37:39 2022] PUT /identity/v3/projects/318d6977a377417faf36dade126e62e3/users/c37e78bd9f704128baf701eba05dca20/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:39.634387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef976f2-43fa-429e-8ae8-636905a0db7a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:39.644156 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef976f2-43fa-429e-8ae8-636905a0db7a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:39.699664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a8389-1a66-415b-94d2-a60ad9c710b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:39.700250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a8389-1a66-415b-94d2-a60ad9c710b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:39.700518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a8389-1a66-415b-94d2-a60ad9c710b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:39.700808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a8389-1a66-415b-94d2-a60ad9c710b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:39.701430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a8389-1a66-415b-94d2-a60ad9c710b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:39.703027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9a8389-1a66-415b-94d2-a60ad9c710b6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:39.712368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1332/2676] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:37:39.717055 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fa867-9d98-4da7-b2bd-55fb676b72d4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:39.727183 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526fa867-9d98-4da7-b2bd-55fb676b72d4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:39.731781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623eb38d-11ff-4a1a-b1d3-9cb148b70cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:39.733059 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623eb38d-11ff-4a1a-b1d3-9cb148b70cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:39.733339 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623eb38d-11ff-4a1a-b1d3-9cb148b70cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:39.733613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623eb38d-11ff-4a1a-b1d3-9cb148b70cc5 admin admin] PATH_INFO: `/v3/projects/318d6977a377417faf36dade126e62e3/users/c37e78bd9f704128baf701eba05dca20/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:39.735798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623eb38d-11ff-4a1a-b1d3-9cb148b70cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=318d6977a377417faf36dade126e62e3, user_id=c37e78bd9f704128baf701eba05dca20, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:39.738517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623eb38d-11ff-4a1a-b1d3-9cb148b70cc5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:39.748567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1345/2677] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:37:39 2022] PUT /identity/v3/projects/318d6977a377417faf36dade126e62e3/users/c37e78bd9f704128baf701eba05dca20/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:39.759726 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b880499-608c-4316-943f-7687e442bb05 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:39.760020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b880499-608c-4316-943f-7687e442bb05 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:39.760287 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b880499-608c-4316-943f-7687e442bb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:39.788046 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0b880499-608c-4316-943f-7687e442bb05 None None] MFA Rules not processed for user `c37e78bd9f704128baf701eba05dca20`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:39.789450 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0b880499-608c-4316-943f-7687e442bb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:39.854059 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:39.858521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1333/2678] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:39 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:39.869271 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623eb38d-11ff-4a1a-b1d3-9cb148b70cc5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:39.870953 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-623eb38d-11ff-4a1a-b1d3-9cb148b70cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:39.881832 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:39.928954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623eb38d-11ff-4a1a-b1d3-9cb148b70cc5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:39.932633 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ce8b46-bc68-45f0-b6ef-de3e12448592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:39.933232 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce8b46-bc68-45f0-b6ef-de3e12448592 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:39.933512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce8b46-bc68-45f0-b6ef-de3e12448592 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:39.933780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce8b46-bc68-45f0-b6ef-de3e12448592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:39.943630 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce8b46-bc68-45f0-b6ef-de3e12448592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:39.945072 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ce8b46-bc68-45f0-b6ef-de3e12448592 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:39.955032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1346/2679] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:37:39 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:37:40.098694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b880499-608c-4316-943f-7687e442bb05 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:40.159548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b880499-608c-4316-943f-7687e442bb05 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:40.163916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:40.164530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:40.164805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:40.165073 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] PATH_INFO: `/v3/users/c13b697eb49c4ad6aaceb47503db4a2d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:40.166209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13b697eb49c4ad6aaceb47503db4a2d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:40.168104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:40.211999 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13b697eb49c4ad6aaceb47503db4a2d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.212633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13b697eb49c4ad6aaceb47503db4a2d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.220099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13b697eb49c4ad6aaceb47503db4a2d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.227091 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13b697eb49c4ad6aaceb47503db4a2d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.238811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13b697eb49c4ad6aaceb47503db4a2d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.244481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1334/2680] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:37:40 2022] DELETE /identity/v3/users/c13b697eb49c4ad6aaceb47503db4a2d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:40.249452 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0147e4b-03dd-44ea-a2a4-8e7425020a66 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:40.249734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0147e4b-03dd-44ea-a2a4-8e7425020a66 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:40.250008 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0147e4b-03dd-44ea-a2a4-8e7425020a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:40.268877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f0147e4b-03dd-44ea-a2a4-8e7425020a66 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:40.270061 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f0147e4b-03dd-44ea-a2a4-8e7425020a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:40.330924 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1347/2681] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:40.340737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:40.342155 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:40.397840 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c34fc-0948-482f-9ec9-12b1d3be7bb6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:40.401484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18094e-6070-44c3-b13a-086a239f105e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:40.402093 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18094e-6070-44c3-b13a-086a239f105e service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:40.402372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18094e-6070-44c3-b13a-086a239f105e service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:40.402641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18094e-6070-44c3-b13a-086a239f105e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:40.411480 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18094e-6070-44c3-b13a-086a239f105e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:40.412819 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18094e-6070-44c3-b13a-086a239f105e service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:40.422034 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1335/2682] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:37:40.685913 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0147e4b-03dd-44ea-a2a4-8e7425020a66 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:40.687522 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f0147e4b-03dd-44ea-a2a4-8e7425020a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:40.699304 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0147e4b-03dd-44ea-a2a4-8e7425020a66 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:40.703882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:40.704578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:40.704863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:40.705131 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] PATH_INFO: `/v3/projects/42bf5b446a8a499dbe79c022a4e6f72c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:40.706276 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] RBAC: Authorizing `identity:delete_project(project_id=42bf5b446a8a499dbe79c022a4e6f72c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:40.708172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:40.754801 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42bf5b446a8a499dbe79c022a4e6f72c', 'request_id': 'req-dc461fab-14ea-459a-9d03-0e672888c56b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.760223 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42bf5b446a8a499dbe79c022a4e6f72c', 'request_id': 'req-dc461fab-14ea-459a-9d03-0e672888c56b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.766358 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42bf5b446a8a499dbe79c022a4e6f72c', 'request_id': 'req-dc461fab-14ea-459a-9d03-0e672888c56b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.778659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] The token cache is being invalidate because project 42bf5b446a8a499dbe79c022a4e6f72c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:37:40.778974 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.779594 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.781097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1348/2683] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:37:40 2022] DELETE /identity/v3/projects/42bf5b446a8a499dbe79c022a4e6f72c => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:40.785076 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18094e-6070-44c3-b13a-086a239f105e service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:40.794437 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18094e-6070-44c3-b13a-086a239f105e service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:40.812960 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:40.850852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8aceb-a958-4839-8df4-f8f04ae63e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:40.851465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8aceb-a958-4839-8df4-f8f04ae63e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:40.851779 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8aceb-a958-4839-8df4-f8f04ae63e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:40.852050 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8aceb-a958-4839-8df4-f8f04ae63e8b admin admin] PATH_INFO: `/v3/users/c37e78bd9f704128baf701eba05dca20` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:40.853139 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8aceb-a958-4839-8df4-f8f04ae63e8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c37e78bd9f704128baf701eba05dca20)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:40.855077 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8aceb-a958-4839-8df4-f8f04ae63e8b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:40.902320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-56e8aceb-a958-4839-8df4-f8f04ae63e8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c37e78bd9f704128baf701eba05dca20'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.902968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-56e8aceb-a958-4839-8df4-f8f04ae63e8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c37e78bd9f704128baf701eba05dca20'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.909645 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-56e8aceb-a958-4839-8df4-f8f04ae63e8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c37e78bd9f704128baf701eba05dca20'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.916529 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-56e8aceb-a958-4839-8df4-f8f04ae63e8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c37e78bd9f704128baf701eba05dca20'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.928817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-56e8aceb-a958-4839-8df4-f8f04ae63e8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c37e78bd9f704128baf701eba05dca20'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:40.934488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1336/2684] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:37:40 2022] DELETE /identity/v3/users/c37e78bd9f704128baf701eba05dca20 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:41.196422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:41.198174 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:41.212597 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc461fab-14ea-459a-9d03-0e672888c56b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:41.248174 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:41.273395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aad066-ba1e-4680-9008-fe3f75b28a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:41.274021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aad066-ba1e-4680-9008-fe3f75b28a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:41.274325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aad066-ba1e-4680-9008-fe3f75b28a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:41.274599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aad066-ba1e-4680-9008-fe3f75b28a04 admin admin] PATH_INFO: `/v3/projects/318d6977a377417faf36dade126e62e3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:41.275722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aad066-ba1e-4680-9008-fe3f75b28a04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=318d6977a377417faf36dade126e62e3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:41.277627 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aad066-ba1e-4680-9008-fe3f75b28a04 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:41.324417 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1aad066-ba1e-4680-9008-fe3f75b28a04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '318d6977a377417faf36dade126e62e3', 'request_id': 'req-d1aad066-ba1e-4680-9008-fe3f75b28a04'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:41.329777 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1aad066-ba1e-4680-9008-fe3f75b28a04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '318d6977a377417faf36dade126e62e3', 'request_id': 'req-d1aad066-ba1e-4680-9008-fe3f75b28a04'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:41.335883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1aad066-ba1e-4680-9008-fe3f75b28a04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '318d6977a377417faf36dade126e62e3', 'request_id': 'req-d1aad066-ba1e-4680-9008-fe3f75b28a04'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:41.348939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1aad066-ba1e-4680-9008-fe3f75b28a04 admin admin] The token cache is being invalidate because project 318d6977a377417faf36dade126e62e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:37:41.349222 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1aad066-ba1e-4680-9008-fe3f75b28a04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:41.349857 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1aad066-ba1e-4680-9008-fe3f75b28a04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:41.351376 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1349/2685] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:37:41 2022] DELETE /identity/v3/projects/318d6977a377417faf36dade126e62e3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:41.356293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57405d-d6e7-43d1-91be-3e9b3af2d347 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:41.356573 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57405d-d6e7-43d1-91be-3e9b3af2d347 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:41.356843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57405d-d6e7-43d1-91be-3e9b3af2d347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:41.376823 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7e57405d-d6e7-43d1-91be-3e9b3af2d347 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:41.378025 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7e57405d-d6e7-43d1-91be-3e9b3af2d347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:41.435348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1337/2686] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:41.443203 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c519c-4dd4-4ecb-b3b5-8ae65db7b479 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:41.443427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c519c-4dd4-4ecb-b3b5-8ae65db7b479 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:41.443699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c519c-4dd4-4ecb-b3b5-8ae65db7b479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:41.463671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-604c519c-4dd4-4ecb-b3b5-8ae65db7b479 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:41.464859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-604c519c-4dd4-4ecb-b3b5-8ae65db7b479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:41.473283 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1350/2687] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:41.477447 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57405d-d6e7-43d1-91be-3e9b3af2d347 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:41.487065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57405d-d6e7-43d1-91be-3e9b3af2d347 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:41.491582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73630ad5-0148-48de-8a55-46a868a5b198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:41.492162 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73630ad5-0148-48de-8a55-46a868a5b198 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:41.492434 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73630ad5-0148-48de-8a55-46a868a5b198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:41.492702 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73630ad5-0148-48de-8a55-46a868a5b198 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:41.493337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73630ad5-0148-48de-8a55-46a868a5b198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:37:41.493634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73630ad5-0148-48de-8a55-46a868a5b198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:41.495226 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73630ad5-0148-48de-8a55-46a868a5b198 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:41.506393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1338/2688] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:37:41.510412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c519c-4dd4-4ecb-b3b5-8ae65db7b479 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:41.519855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c519c-4dd4-4ecb-b3b5-8ae65db7b479 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:41.524482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfb70e-71d1-42fd-973e-f1500d9f2a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:41.525064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfb70e-71d1-42fd-973e-f1500d9f2a35 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:41.525340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfb70e-71d1-42fd-973e-f1500d9f2a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:41.525612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bfb70e-71d1-42fd-973e-f1500d9f2a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:41.526277 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfb70e-71d1-42fd-973e-f1500d9f2a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:41.528156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bfb70e-71d1-42fd-973e-f1500d9f2a35 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:41.556158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1351/2689] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:37:41 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:37:41.559919 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73630ad5-0148-48de-8a55-46a868a5b198 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:41.569531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73630ad5-0148-48de-8a55-46a868a5b198 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:41.595507 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:41.625038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c80c2-df14-48b3-a270-f404b03a87dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:41.625621 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c80c2-df14-48b3-a270-f404b03a87dd admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:41.625892 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c80c2-df14-48b3-a270-f404b03a87dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:41.626191 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c80c2-df14-48b3-a270-f404b03a87dd admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:41.626810 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c80c2-df14-48b3-a270-f404b03a87dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:41.628705 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c80c2-df14-48b3-a270-f404b03a87dd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:41.649440 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1339/2690] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:37:41 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:37:41.653663 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfb70e-71d1-42fd-973e-f1500d9f2a35 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:41.664477 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bfb70e-71d1-42fd-973e-f1500d9f2a35 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:41.669172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa867bd-f274-49aa-938d-6f097d5313d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:41.669802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa867bd-f274-49aa-938d-6f097d5313d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:41.670110 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa867bd-f274-49aa-938d-6f097d5313d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:41.670380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa867bd-f274-49aa-938d-6f097d5313d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:41.671019 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa867bd-f274-49aa-938d-6f097d5313d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:41.672593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa867bd-f274-49aa-938d-6f097d5313d8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:41.682598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1352/2691] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:37:41.686417 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c80c2-df14-48b3-a270-f404b03a87dd admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:41.696316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c80c2-df14-48b3-a270-f404b03a87dd admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:41.701288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac032bb-055a-466d-84e6-d111bad8f827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:41.701866 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac032bb-055a-466d-84e6-d111bad8f827 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:41.702167 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac032bb-055a-466d-84e6-d111bad8f827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:41.702432 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac032bb-055a-466d-84e6-d111bad8f827 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:41.703056 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac032bb-055a-466d-84e6-d111bad8f827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:41.704627 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac032bb-055a-466d-84e6-d111bad8f827 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:41.714902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1340/2692] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:37:41.719438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa867bd-f274-49aa-938d-6f097d5313d8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:41.729298 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa867bd-f274-49aa-938d-6f097d5313d8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:41.734013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2606c-14a5-4763-b2b3-31ce3b52039e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:41.734681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2606c-14a5-4763-b2b3-31ce3b52039e admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:41.734960 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2606c-14a5-4763-b2b3-31ce3b52039e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:41.735231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2606c-14a5-4763-b2b3-31ce3b52039e admin admin] PATH_INFO: `/v3/projects/2f8c3684c1cc4bbd86f12850c1e4142d/users/4f947c77b28840418ce92b6815410268/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:41.750409 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2606c-14a5-4763-b2b3-31ce3b52039e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f8c3684c1cc4bbd86f12850c1e4142d, user_id=4f947c77b28840418ce92b6815410268, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:41.753072 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2606c-14a5-4763-b2b3-31ce3b52039e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:41.766685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1353/2693] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:37:41 2022] PUT /identity/v3/projects/2f8c3684c1cc4bbd86f12850c1e4142d/users/4f947c77b28840418ce92b6815410268/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:41.777011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874f488-5ef0-4f04-b980-e3cc607f78b4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:41.777291 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874f488-5ef0-4f04-b980-e3cc607f78b4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:41.777558 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6874f488-5ef0-4f04-b980-e3cc607f78b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:41.804083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6874f488-5ef0-4f04-b980-e3cc607f78b4 None None] MFA Rules not processed for user `4f947c77b28840418ce92b6815410268`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:41.805279 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6874f488-5ef0-4f04-b980-e3cc607f78b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:41.870641 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1341/2694] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:41.875908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2606c-14a5-4763-b2b3-31ce3b52039e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:41.936021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2606c-14a5-4763-b2b3-31ce3b52039e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:41.940720 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfa011-abd9-4976-8932-e7ef41fc76d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:41.941310 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfa011-abd9-4976-8932-e7ef41fc76d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:41.941587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfa011-abd9-4976-8932-e7ef41fc76d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:41.941857 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfa011-abd9-4976-8932-e7ef41fc76d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:41.942513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfa011-abd9-4976-8932-e7ef41fc76d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:41.944408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfa011-abd9-4976-8932-e7ef41fc76d0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:41.949627 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:41.973014 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1354/2695] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:37:41 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:37:41.977208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874f488-5ef0-4f04-b980-e3cc607f78b4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:41.988023 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6874f488-5ef0-4f04-b980-e3cc607f78b4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:42.042123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b5d2f-a09b-4f6b-ac34-915a054fb7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:42.042713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b5d2f-a09b-4f6b-ac34-915a054fb7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:42.042987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b5d2f-a09b-4f6b-ac34-915a054fb7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:42.043252 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b5d2f-a09b-4f6b-ac34-915a054fb7a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:42.043872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b5d2f-a09b-4f6b-ac34-915a054fb7a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:42.045753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b5d2f-a09b-4f6b-ac34-915a054fb7a1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:42.067522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1342/2696] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:37:41 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:37:42.071564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfa011-abd9-4976-8932-e7ef41fc76d0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:42.082398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfa011-abd9-4976-8932-e7ef41fc76d0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:42.086892 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9309e-44b3-4098-baff-22cb0a934cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:42.087470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9309e-44b3-4098-baff-22cb0a934cbf admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:42.087742 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9309e-44b3-4098-baff-22cb0a934cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:42.088010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9309e-44b3-4098-baff-22cb0a934cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:42.088645 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9309e-44b3-4098-baff-22cb0a934cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:42.090248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be9309e-44b3-4098-baff-22cb0a934cbf admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:42.100866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1355/2697] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:37:42.105260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b5d2f-a09b-4f6b-ac34-915a054fb7a1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:42.119961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b5d2f-a09b-4f6b-ac34-915a054fb7a1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:42.124972 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4c29c-cdde-4a30-b694-8a11e3c4d9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:42.125602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4c29c-cdde-4a30-b694-8a11e3c4d9ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:42.125874 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4c29c-cdde-4a30-b694-8a11e3c4d9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:42.126168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4c29c-cdde-4a30-b694-8a11e3c4d9ce admin admin] PATH_INFO: `/v3/projects/208488221888472ea0e35475114d0d05/users/3c2eb4a56f8f4b62a7410246861d318c/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:42.142640 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4c29c-cdde-4a30-b694-8a11e3c4d9ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=208488221888472ea0e35475114d0d05, user_id=3c2eb4a56f8f4b62a7410246861d318c, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:42.145355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4c29c-cdde-4a30-b694-8a11e3c4d9ce admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:42.158449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1343/2698] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:37:42 2022] PUT /identity/v3/projects/208488221888472ea0e35475114d0d05/users/3c2eb4a56f8f4b62a7410246861d318c/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:37:42.162708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9309e-44b3-4098-baff-22cb0a934cbf admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:42.173021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9309e-44b3-4098-baff-22cb0a934cbf admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:42.226726 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc17ab-0175-4382-8ac0-147ba8ad9bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:42.227243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc17ab-0175-4382-8ac0-147ba8ad9bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:42.227516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc17ab-0175-4382-8ac0-147ba8ad9bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:42.227783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc17ab-0175-4382-8ac0-147ba8ad9bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:42.228408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc17ab-0175-4382-8ac0-147ba8ad9bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:42.229982 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc17ab-0175-4382-8ac0-147ba8ad9bd0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:42.240106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1356/2699] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:37:42.244253 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4c29c-cdde-4a30-b694-8a11e3c4d9ce admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:42.250372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:42.256885 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4c29c-cdde-4a30-b694-8a11e3c4d9ce admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:42.261605 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a7892-f55a-4538-8b90-c90793fed8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:42.262278 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a7892-f55a-4538-8b90-c90793fed8ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:42.262552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a7892-f55a-4538-8b90-c90793fed8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:42.262819 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a7892-f55a-4538-8b90-c90793fed8ef admin admin] PATH_INFO: `/v3/projects/208488221888472ea0e35475114d0d05/users/3c2eb4a56f8f4b62a7410246861d318c/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:42.264720 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a7892-f55a-4538-8b90-c90793fed8ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=208488221888472ea0e35475114d0d05, user_id=3c2eb4a56f8f4b62a7410246861d318c, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:42.267349 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a7892-f55a-4538-8b90-c90793fed8ef admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:42.277224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1344/2700] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:37:42 2022] PUT /identity/v3/projects/208488221888472ea0e35475114d0d05/users/3c2eb4a56f8f4b62a7410246861d318c/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:42.288406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2f2c8-5932-44c5-9155-bd1d7d3c435c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:42.288694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2f2c8-5932-44c5-9155-bd1d7d3c435c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:42.288965 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2f2c8-5932-44c5-9155-bd1d7d3c435c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:42.317692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-bee2f2c8-5932-44c5-9155-bd1d7d3c435c None None] MFA Rules not processed for user `3c2eb4a56f8f4b62a7410246861d318c`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:42.318941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-bee2f2c8-5932-44c5-9155-bd1d7d3c435c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:42.384352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1357/2701] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:42.398307 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a7892-f55a-4538-8b90-c90793fed8ef admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:42.400207 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-974a7892-f55a-4538-8b90-c90793fed8ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:42.476510 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a7892-f55a-4538-8b90-c90793fed8ef admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:42.480169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61775c-352f-447f-bc2a-ce0522361e98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:42.480692 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61775c-352f-447f-bc2a-ce0522361e98 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:42.480966 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61775c-352f-447f-bc2a-ce0522361e98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:42.481234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61775c-352f-447f-bc2a-ce0522361e98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:42.496484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61775c-352f-447f-bc2a-ce0522361e98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:42.498595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61775c-352f-447f-bc2a-ce0522361e98 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:42.508819 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1345/2702] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:37:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:37:42.611532 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2f2c8-5932-44c5-9155-bd1d7d3c435c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:42.620944 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2f2c8-5932-44c5-9155-bd1d7d3c435c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:42.624492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe8b5b-a8cb-4f4b-bd8c-576fd5bb7a0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:42.625080 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe8b5b-a8cb-4f4b-bd8c-576fd5bb7a0d service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:42.625353 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe8b5b-a8cb-4f4b-bd8c-576fd5bb7a0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:42.625622 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe8b5b-a8cb-4f4b-bd8c-576fd5bb7a0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:42.678398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:42.680708 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe8b5b-a8cb-4f4b-bd8c-576fd5bb7a0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:42.682945 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe8b5b-a8cb-4f4b-bd8c-576fd5bb7a0d service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:42.692253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1358/2703] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:37:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:37:45.285778 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61775c-352f-447f-bc2a-ce0522361e98 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:45.346596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61775c-352f-447f-bc2a-ce0522361e98 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:45.351065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:45.351662 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:45.351932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:45.352199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] PATH_INFO: `/v3/users/4f947c77b28840418ce92b6815410268` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:45.353288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f947c77b28840418ce92b6815410268)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:45.355214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:45.406751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f947c77b28840418ce92b6815410268'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:45.407422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f947c77b28840418ce92b6815410268'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:45.414183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f947c77b28840418ce92b6815410268'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:45.421368 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f947c77b28840418ce92b6815410268'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:45.432977 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f947c77b28840418ce92b6815410268'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:45.438704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1346/2704] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:37:45 2022] DELETE /identity/v3/users/4f947c77b28840418ce92b6815410268 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:45.443195 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef85c06-dc4d-4cff-882d-71047d0c849b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:45.443681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef85c06-dc4d-4cff-882d-71047d0c849b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:45.443798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef85c06-dc4d-4cff-882d-71047d0c849b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:45.462705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2ef85c06-dc4d-4cff-882d-71047d0c849b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:45.463899 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2ef85c06-dc4d-4cff-882d-71047d0c849b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:45.522695 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1359/2705] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:45.532310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:45.533705 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:45.547087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:45.591929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbd8dc-f6e5-4e19-900d-7046a518148c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:45.595564 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f650f8a-f1e9-4d20-aa5f-e5e5d351abb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:45.596158 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f650f8a-f1e9-4d20-aa5f-e5e5d351abb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:45.596436 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f650f8a-f1e9-4d20-aa5f-e5e5d351abb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:45.596707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f650f8a-f1e9-4d20-aa5f-e5e5d351abb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:45.605569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f650f8a-f1e9-4d20-aa5f-e5e5d351abb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:45.607010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f650f8a-f1e9-4d20-aa5f-e5e5d351abb4 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:45.616543 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1347/2706] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:37:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:37:45.891820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef85c06-dc4d-4cff-882d-71047d0c849b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:45.893395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2ef85c06-dc4d-4cff-882d-71047d0c849b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:45.907218 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef85c06-dc4d-4cff-882d-71047d0c849b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:45.912805 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:45.913573 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:45.913976 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:45.914418 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] PATH_INFO: `/v3/projects/2f8c3684c1cc4bbd86f12850c1e4142d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:45.915836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f8c3684c1cc4bbd86f12850c1e4142d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:45.918205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:45.965184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f8c3684c1cc4bbd86f12850c1e4142d', 'request_id': 'req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:45.970544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f8c3684c1cc4bbd86f12850c1e4142d', 'request_id': 'req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:45.976854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f8c3684c1cc4bbd86f12850c1e4142d', 'request_id': 'req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:45.989820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] The token cache is being invalidate because project 2f8c3684c1cc4bbd86f12850c1e4142d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:37:45.990160 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:45.990780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:45.992249 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1360/2707] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:37:45 2022] DELETE /identity/v3/projects/2f8c3684c1cc4bbd86f12850c1e4142d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:45.996289 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f650f8a-f1e9-4d20-aa5f-e5e5d351abb4 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:46.006370 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f650f8a-f1e9-4d20-aa5f-e5e5d351abb4 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:46.061769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a106ca5-86bf-499e-8a43-7cfd1e3031b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:46.062345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a106ca5-86bf-499e-8a43-7cfd1e3031b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:46.062622 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a106ca5-86bf-499e-8a43-7cfd1e3031b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:46.062890 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a106ca5-86bf-499e-8a43-7cfd1e3031b9 admin admin] PATH_INFO: `/v3/users/3c2eb4a56f8f4b62a7410246861d318c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:46.063988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a106ca5-86bf-499e-8a43-7cfd1e3031b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c2eb4a56f8f4b62a7410246861d318c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:46.065885 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a106ca5-86bf-499e-8a43-7cfd1e3031b9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:46.109233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1a106ca5-86bf-499e-8a43-7cfd1e3031b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c2eb4a56f8f4b62a7410246861d318c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:46.109883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1a106ca5-86bf-499e-8a43-7cfd1e3031b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c2eb4a56f8f4b62a7410246861d318c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:46.116781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1a106ca5-86bf-499e-8a43-7cfd1e3031b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c2eb4a56f8f4b62a7410246861d318c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:46.124098 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1a106ca5-86bf-499e-8a43-7cfd1e3031b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c2eb4a56f8f4b62a7410246861d318c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:46.135737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1a106ca5-86bf-499e-8a43-7cfd1e3031b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c2eb4a56f8f4b62a7410246861d318c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:46.141548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1348/2708] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:37:45 2022] DELETE /identity/v3/users/3c2eb4a56f8f4b62a7410246861d318c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:46.405432 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:46.406887 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:46.420795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4b30e-6027-4a0e-8e65-4f9bcc1d53ce admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:46.476058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f3c6a-6325-477e-9663-c75a77025c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:46.476693 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f3c6a-6325-477e-9663-c75a77025c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:46.476940 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f3c6a-6325-477e-9663-c75a77025c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:46.477227 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f3c6a-6325-477e-9663-c75a77025c3b admin admin] PATH_INFO: `/v3/projects/208488221888472ea0e35475114d0d05` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:46.478372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f3c6a-6325-477e-9663-c75a77025c3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=208488221888472ea0e35475114d0d05)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:46.480254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f3c6a-6325-477e-9663-c75a77025c3b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:46.504482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:46.528016 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-703f3c6a-6325-477e-9663-c75a77025c3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '208488221888472ea0e35475114d0d05', 'request_id': 'req-703f3c6a-6325-477e-9663-c75a77025c3b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:46.533908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-703f3c6a-6325-477e-9663-c75a77025c3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '208488221888472ea0e35475114d0d05', 'request_id': 'req-703f3c6a-6325-477e-9663-c75a77025c3b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:46.539925 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-703f3c6a-6325-477e-9663-c75a77025c3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '208488221888472ea0e35475114d0d05', 'request_id': 'req-703f3c6a-6325-477e-9663-c75a77025c3b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:46.552195 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-703f3c6a-6325-477e-9663-c75a77025c3b admin admin] The token cache is being invalidate because project 208488221888472ea0e35475114d0d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:37:46.552482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-703f3c6a-6325-477e-9663-c75a77025c3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:46.553095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-703f3c6a-6325-477e-9663-c75a77025c3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:37:46.554603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1361/2709] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:37:46 2022] DELETE /identity/v3/projects/208488221888472ea0e35475114d0d05 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:37:46.559402 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d5b59-dc29-4b4f-8269-fac74e9d1269 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:46.559894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d5b59-dc29-4b4f-8269-fac74e9d1269 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:46.560000 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d5b59-dc29-4b4f-8269-fac74e9d1269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:46.578968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-de9d5b59-dc29-4b4f-8269-fac74e9d1269 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:46.580078 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-de9d5b59-dc29-4b4f-8269-fac74e9d1269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:46.609449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:46.641270 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1349/2710] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:46.648926 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77027c63-2b79-4811-8d92-5301e535f063 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:46.649209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77027c63-2b79-4811-8d92-5301e535f063 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:46.649480 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77027c63-2b79-4811-8d92-5301e535f063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:46.669101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-77027c63-2b79-4811-8d92-5301e535f063 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:46.670422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-77027c63-2b79-4811-8d92-5301e535f063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:46.678502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1362/2711] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:46.682764 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d5b59-dc29-4b4f-8269-fac74e9d1269 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:46.692400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d5b59-dc29-4b4f-8269-fac74e9d1269 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:46.696895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5fe7f-7df6-4e6c-a685-6f0a0add9bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:46.697477 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5fe7f-7df6-4e6c-a685-6f0a0add9bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:46.697751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5fe7f-7df6-4e6c-a685-6f0a0add9bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:46.698022 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5fe7f-7df6-4e6c-a685-6f0a0add9bd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:46.698691 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5fe7f-7df6-4e6c-a685-6f0a0add9bd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:37:46.698987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5fe7f-7df6-4e6c-a685-6f0a0add9bd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:46.700555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5fe7f-7df6-4e6c-a685-6f0a0add9bd4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:46.712370 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1350/2712] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:37:46.717527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77027c63-2b79-4811-8d92-5301e535f063 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:46.726817 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77027c63-2b79-4811-8d92-5301e535f063 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:46.731465 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8e1fe-269c-41d6-936b-805ca1a4a37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:46.732053 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8e1fe-269c-41d6-936b-805ca1a4a37b admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:46.732333 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8e1fe-269c-41d6-936b-805ca1a4a37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:46.732602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8e1fe-269c-41d6-936b-805ca1a4a37b admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:46.733232 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8e1fe-269c-41d6-936b-805ca1a4a37b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:46.735157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8e1fe-269c-41d6-936b-805ca1a4a37b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:46.761563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1363/2713] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:37:46 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:37:46.766403 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5fe7f-7df6-4e6c-a685-6f0a0add9bd4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:46.776554 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5fe7f-7df6-4e6c-a685-6f0a0add9bd4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:46.832120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1be70-8373-499e-83cf-156138f4bb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:46.832642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1be70-8373-499e-83cf-156138f4bb1f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:46.832918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1be70-8373-499e-83cf-156138f4bb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:46.833188 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1be70-8373-499e-83cf-156138f4bb1f admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:46.833806 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1be70-8373-499e-83cf-156138f4bb1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:46.835761 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1be70-8373-499e-83cf-156138f4bb1f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:46.856616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1351/2714] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:37:46 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:37:46.861153 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8e1fe-269c-41d6-936b-805ca1a4a37b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:46.871169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8e1fe-269c-41d6-936b-805ca1a4a37b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:46.875806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dadc22-ce6e-43f4-b186-c5eaf95188d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:46.876392 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dadc22-ce6e-43f4-b186-c5eaf95188d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:46.876671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dadc22-ce6e-43f4-b186-c5eaf95188d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:46.876942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dadc22-ce6e-43f4-b186-c5eaf95188d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:46.877576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dadc22-ce6e-43f4-b186-c5eaf95188d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:46.879195 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dadc22-ce6e-43f4-b186-c5eaf95188d3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:46.889683 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1364/2715] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:37:46.893740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1be70-8373-499e-83cf-156138f4bb1f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:46.903413 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1be70-8373-499e-83cf-156138f4bb1f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:46.908614 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f98174-3f6f-47b6-8510-a836438ffdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:46.909188 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f98174-3f6f-47b6-8510-a836438ffdd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:46.909457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f98174-3f6f-47b6-8510-a836438ffdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:46.909722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f98174-3f6f-47b6-8510-a836438ffdd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:46.910374 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f98174-3f6f-47b6-8510-a836438ffdd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:46.911920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f98174-3f6f-47b6-8510-a836438ffdd4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:46.922177 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1352/2716] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:37:46.926018 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dadc22-ce6e-43f4-b186-c5eaf95188d3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:46.935737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dadc22-ce6e-43f4-b186-c5eaf95188d3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:46.940553 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d9e08-cba0-46ea-b35d-ba9f307808bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:46.941225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d9e08-cba0-46ea-b35d-ba9f307808bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:46.941504 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d9e08-cba0-46ea-b35d-ba9f307808bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:46.941774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d9e08-cba0-46ea-b35d-ba9f307808bb admin admin] PATH_INFO: `/v3/projects/9140b3851471414a91c52a599dab29d9/users/656cb3bcf5114a018d0cd7c4f25c3aae/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:46.956621 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d9e08-cba0-46ea-b35d-ba9f307808bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9140b3851471414a91c52a599dab29d9, user_id=656cb3bcf5114a018d0cd7c4f25c3aae, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:46.959433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d9e08-cba0-46ea-b35d-ba9f307808bb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:46.973384 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1365/2717] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:37:46 2022] PUT /identity/v3/projects/9140b3851471414a91c52a599dab29d9/users/656cb3bcf5114a018d0cd7c4f25c3aae/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:46.978351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c0aff-1612-4b4c-8055-8342b26807f7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:46.978631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c0aff-1612-4b4c-8055-8342b26807f7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:46.978903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c0aff-1612-4b4c-8055-8342b26807f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:46.998240 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4a7c0aff-1612-4b4c-8055-8342b26807f7 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:46.999440 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4a7c0aff-1612-4b4c-8055-8342b26807f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:47.063292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1353/2718] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:47.073162 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d9e08-cba0-46ea-b35d-ba9f307808bb admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:47.074895 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-075d9e08-cba0-46ea-b35d-ba9f307808bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:47.131790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d9e08-cba0-46ea-b35d-ba9f307808bb admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:47.135408 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54224ada-86b0-49a3-b665-a655fe423aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:47.136013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54224ada-86b0-49a3-b665-a655fe423aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:47.136286 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54224ada-86b0-49a3-b665-a655fe423aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:47.136554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54224ada-86b0-49a3-b665-a655fe423aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:47.146045 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54224ada-86b0-49a3-b665-a655fe423aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:47.147499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54224ada-86b0-49a3-b665-a655fe423aa9 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:47.156789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1366/2719] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 27 17:37:47.781733 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e3041-2201-4ed1-9e6b-572f14ba6ad9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:47.782249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e3041-2201-4ed1-9e6b-572f14ba6ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:47.782372 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e3041-2201-4ed1-9e6b-572f14ba6ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:47.809493 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-cd0e3041-2201-4ed1-9e6b-572f14ba6ad9 None None] MFA Rules not processed for user `656cb3bcf5114a018d0cd7c4f25c3aae`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:47.810809 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cd0e3041-2201-4ed1-9e6b-572f14ba6ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:47.838584 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:47.876039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1354/2720] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:47.880684 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54224ada-86b0-49a3-b665-a655fe423aa9 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:47.890352 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54224ada-86b0-49a3-b665-a655fe423aa9 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:47.895004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67c790-3216-4d9a-9905-fe336cd80a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:47.895617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67c790-3216-4d9a-9905-fe336cd80a48 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:47.895889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67c790-3216-4d9a-9905-fe336cd80a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:47.896159 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67c790-3216-4d9a-9905-fe336cd80a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:47.896791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67c790-3216-4d9a-9905-fe336cd80a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:47.898733 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f67c790-3216-4d9a-9905-fe336cd80a48 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:47.928651 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1367/2721] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:37:47 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:37:47.932293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e3041-2201-4ed1-9e6b-572f14ba6ad9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:47.942807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e3041-2201-4ed1-9e6b-572f14ba6ad9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:47.997913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdc6f8-4cdd-4c33-90e5-fbb4d25f2904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:47.998540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdc6f8-4cdd-4c33-90e5-fbb4d25f2904 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:47.998818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdc6f8-4cdd-4c33-90e5-fbb4d25f2904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:47.999085 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdc6f8-4cdd-4c33-90e5-fbb4d25f2904 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:47.999698 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdc6f8-4cdd-4c33-90e5-fbb4d25f2904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:48.001586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdc6f8-4cdd-4c33-90e5-fbb4d25f2904 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:48.021670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1355/2722] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:37:47 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:37:48.025785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67c790-3216-4d9a-9905-fe336cd80a48 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:48.036592 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67c790-3216-4d9a-9905-fe336cd80a48 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:48.041172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00353450-9aab-4b9f-8dd9-33dfa90f5c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:48.041758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00353450-9aab-4b9f-8dd9-33dfa90f5c16 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:48.042035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00353450-9aab-4b9f-8dd9-33dfa90f5c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:48.042342 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00353450-9aab-4b9f-8dd9-33dfa90f5c16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:48.042980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00353450-9aab-4b9f-8dd9-33dfa90f5c16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:48.045142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00353450-9aab-4b9f-8dd9-33dfa90f5c16 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:48.055497 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1368/2723] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:37:48.059552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdc6f8-4cdd-4c33-90e5-fbb4d25f2904 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:48.069292 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdc6f8-4cdd-4c33-90e5-fbb4d25f2904 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:48.073831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0685e-bbd2-4095-9c92-cf32c94b96d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:48.074497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0685e-bbd2-4095-9c92-cf32c94b96d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:48.074773 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0685e-bbd2-4095-9c92-cf32c94b96d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:48.075044 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0685e-bbd2-4095-9c92-cf32c94b96d8 admin admin] PATH_INFO: `/v3/projects/7a11ad72b5e845faacaea4d2361a1b9c/users/91eca21e73304d94b1f71fd4ed49afeb/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:48.089218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0685e-bbd2-4095-9c92-cf32c94b96d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a11ad72b5e845faacaea4d2361a1b9c, user_id=91eca21e73304d94b1f71fd4ed49afeb, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:48.092380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0685e-bbd2-4095-9c92-cf32c94b96d8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:48.107957 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1356/2724] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:37:48 2022] PUT /identity/v3/projects/7a11ad72b5e845faacaea4d2361a1b9c/users/91eca21e73304d94b1f71fd4ed49afeb/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:48.112360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00353450-9aab-4b9f-8dd9-33dfa90f5c16 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:48.122187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00353450-9aab-4b9f-8dd9-33dfa90f5c16 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:48.133394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:48.180550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5039a88-5074-4cfc-bb7a-ae5048ff8b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:48.181187 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039a88-5074-4cfc-bb7a-ae5048ff8b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:48.181529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039a88-5074-4cfc-bb7a-ae5048ff8b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:48.181860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039a88-5074-4cfc-bb7a-ae5048ff8b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:48.185540 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5039a88-5074-4cfc-bb7a-ae5048ff8b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:48.185540 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5039a88-5074-4cfc-bb7a-ae5048ff8b04 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:48.195528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1369/2725] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:37:48.201323 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0685e-bbd2-4095-9c92-cf32c94b96d8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:48.213095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0685e-bbd2-4095-9c92-cf32c94b96d8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:48.218752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25f3cc-23a8-40f6-9fa8-f767ff69e9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:48.219582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25f3cc-23a8-40f6-9fa8-f767ff69e9c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:48.219886 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25f3cc-23a8-40f6-9fa8-f767ff69e9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:48.220169 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25f3cc-23a8-40f6-9fa8-f767ff69e9c0 admin admin] PATH_INFO: `/v3/projects/7a11ad72b5e845faacaea4d2361a1b9c/users/91eca21e73304d94b1f71fd4ed49afeb/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:48.222054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25f3cc-23a8-40f6-9fa8-f767ff69e9c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a11ad72b5e845faacaea4d2361a1b9c, user_id=91eca21e73304d94b1f71fd4ed49afeb, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:48.224674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25f3cc-23a8-40f6-9fa8-f767ff69e9c0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:48.235290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1357/2726] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:37:48 2022] PUT /identity/v3/projects/7a11ad72b5e845faacaea4d2361a1b9c/users/91eca21e73304d94b1f71fd4ed49afeb/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:37:48.862390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997242e-230d-4d5c-8623-f3c642fa6738 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:48.862669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997242e-230d-4d5c-8623-f3c642fa6738 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:48.862943 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997242e-230d-4d5c-8623-f3c642fa6738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:48.891548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9997242e-230d-4d5c-8623-f3c642fa6738 None None] MFA Rules not processed for user `91eca21e73304d94b1f71fd4ed49afeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:37:48.892809 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9997242e-230d-4d5c-8623-f3c642fa6738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:48.962226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1370/2727] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:37:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:37:48.973936 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25f3cc-23a8-40f6-9fa8-f767ff69e9c0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:48.975768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9b25f3cc-23a8-40f6-9fa8-f767ff69e9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:49.040033 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25f3cc-23a8-40f6-9fa8-f767ff69e9c0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:49.043837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b197368-4dda-42c7-9815-00d9418e2465 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:49.044945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b197368-4dda-42c7-9815-00d9418e2465 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:49.045217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b197368-4dda-42c7-9815-00d9418e2465 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:49.045481 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b197368-4dda-42c7-9815-00d9418e2465 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:49.083308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:37:49.108417 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b197368-4dda-42c7-9815-00d9418e2465 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:49.109849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b197368-4dda-42c7-9815-00d9418e2465 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:49.118955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1358/2728] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:37:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:37:50.478430 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997242e-230d-4d5c-8623-f3c642fa6738 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:37:50.484457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9997242e-230d-4d5c-8623-f3c642fa6738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:37:50.496951 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997242e-230d-4d5c-8623-f3c642fa6738 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:37:50.501077 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b639d-644d-4dd7-a1e3-b950084558bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:37:50.501665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b639d-644d-4dd7-a1e3-b950084558bf service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:37:50.501937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b639d-644d-4dd7-a1e3-b950084558bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:37:50.502239 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b639d-644d-4dd7-a1e3-b950084558bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:37:50.510959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b639d-644d-4dd7-a1e3-b950084558bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:37:50.512366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b639d-644d-4dd7-a1e3-b950084558bf service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:37:50.521471 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1371/2729] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:37:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:37:59.115001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=11 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:00.517391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=62 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:02.909848 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b197368-4dda-42c7-9815-00d9418e2465 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:02.911304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2b197368-4dda-42c7-9815-00d9418e2465 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:02.993774 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b197368-4dda-42c7-9815-00d9418e2465 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:02.998548 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27392ed4-fa29-4e6c-9672-534590320d1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:02.999631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27392ed4-fa29-4e6c-9672-534590320d1a service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:02.999988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27392ed4-fa29-4e6c-9672-534590320d1a service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:02.999988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27392ed4-fa29-4e6c-9672-534590320d1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:03.001840 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-27392ed4-fa29-4e6c-9672-534590320d1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:03.013893 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27392ed4-fa29-4e6c-9672-534590320d1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:03.015577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27392ed4-fa29-4e6c-9672-534590320d1a service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:03.026184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1359/2730] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:38:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:38:03.390009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b639d-644d-4dd7-a1e3-b950084558bf service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:03.405931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b639d-644d-4dd7-a1e3-b950084558bf service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:03.411784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b436fdc-cef9-444f-ab04-d7282c47b3a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:03.412368 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b436fdc-cef9-444f-ab04-d7282c47b3a5 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:03.412640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b436fdc-cef9-444f-ab04-d7282c47b3a5 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:03.412909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b436fdc-cef9-444f-ab04-d7282c47b3a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:03.414681 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1b436fdc-cef9-444f-ab04-d7282c47b3a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:03.426039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b436fdc-cef9-444f-ab04-d7282c47b3a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:03.427625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b436fdc-cef9-444f-ab04-d7282c47b3a5 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:03.437713 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1372/2731] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:38:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:38:03.453026 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27392ed4-fa29-4e6c-9672-534590320d1a service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:03.516740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27392ed4-fa29-4e6c-9672-534590320d1a service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:03.521267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:03.521889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:03.522203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:03.522477 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] PATH_INFO: `/v3/users/656cb3bcf5114a018d0cd7c4f25c3aae` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:03.523594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=656cb3bcf5114a018d0cd7c4f25c3aae)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:03.525502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:03.568477 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '656cb3bcf5114a018d0cd7c4f25c3aae'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:03.569111 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '656cb3bcf5114a018d0cd7c4f25c3aae'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:03.575883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '656cb3bcf5114a018d0cd7c4f25c3aae'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:03.583056 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '656cb3bcf5114a018d0cd7c4f25c3aae'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:03.594919 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '656cb3bcf5114a018d0cd7c4f25c3aae'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:03.600700 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1360/2732] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:38:03 2022] DELETE /identity/v3/users/656cb3bcf5114a018d0cd7c4f25c3aae => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:03.769441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b436fdc-cef9-444f-ab04-d7282c47b3a5 service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:03.782962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b436fdc-cef9-444f-ab04-d7282c47b3a5 service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:03.839517 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:03.840147 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:03.840422 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:03.840689 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] PATH_INFO: `/v3/projects/9140b3851471414a91c52a599dab29d9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:03.841826 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] RBAC: Authorizing `identity:delete_project(project_id=9140b3851471414a91c52a599dab29d9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:03.843789 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:03.888576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9140b3851471414a91c52a599dab29d9', 'request_id': 'req-45af66ed-47d2-40b5-acce-b70bf28fc6db'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:03.893848 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9140b3851471414a91c52a599dab29d9', 'request_id': 'req-45af66ed-47d2-40b5-acce-b70bf28fc6db'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:03.899855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9140b3851471414a91c52a599dab29d9', 'request_id': 'req-45af66ed-47d2-40b5-acce-b70bf28fc6db'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:03.911464 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] The token cache is being invalidate because project 9140b3851471414a91c52a599dab29d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:38:03.911772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:03.912350 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:03.913784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1373/2733] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:38:03 2022] DELETE /identity/v3/projects/9140b3851471414a91c52a599dab29d9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:03.917849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:03.919500 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:03.933060 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dccbc6-b1f1-4878-99ce-b7b768839979 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:03.937563 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:03.995301 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80440eb5-6038-45d7-8b85-0094f0697f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:03.995925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80440eb5-6038-45d7-8b85-0094f0697f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:03.996204 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80440eb5-6038-45d7-8b85-0094f0697f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:03.996473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80440eb5-6038-45d7-8b85-0094f0697f34 admin admin] PATH_INFO: `/v3/users/91eca21e73304d94b1f71fd4ed49afeb` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:03.997600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80440eb5-6038-45d7-8b85-0094f0697f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91eca21e73304d94b1f71fd4ed49afeb)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:03.999539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80440eb5-6038-45d7-8b85-0094f0697f34 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:04.043668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-80440eb5-6038-45d7-8b85-0094f0697f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91eca21e73304d94b1f71fd4ed49afeb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:04.044302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-80440eb5-6038-45d7-8b85-0094f0697f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91eca21e73304d94b1f71fd4ed49afeb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:04.052189 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-80440eb5-6038-45d7-8b85-0094f0697f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91eca21e73304d94b1f71fd4ed49afeb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:04.062174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-80440eb5-6038-45d7-8b85-0094f0697f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91eca21e73304d94b1f71fd4ed49afeb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:04.075692 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-80440eb5-6038-45d7-8b85-0094f0697f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91eca21e73304d94b1f71fd4ed49afeb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:04.081585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1361/2734] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:38:03 2022] DELETE /identity/v3/users/91eca21e73304d94b1f71fd4ed49afeb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:04.514576 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:04.516438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:04.529751 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af66ed-47d2-40b5-acce-b70bf28fc6db admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:04.587272 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716fad3-462e-41cc-85c3-5680c1854f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:04.587902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716fad3-462e-41cc-85c3-5680c1854f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:04.588178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716fad3-462e-41cc-85c3-5680c1854f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:04.588449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716fad3-462e-41cc-85c3-5680c1854f90 admin admin] PATH_INFO: `/v3/projects/7a11ad72b5e845faacaea4d2361a1b9c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:04.589563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716fad3-462e-41cc-85c3-5680c1854f90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a11ad72b5e845faacaea4d2361a1b9c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:04.591501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716fad3-462e-41cc-85c3-5680c1854f90 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:04.614369 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:04.640357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c716fad3-462e-41cc-85c3-5680c1854f90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a11ad72b5e845faacaea4d2361a1b9c', 'request_id': 'req-c716fad3-462e-41cc-85c3-5680c1854f90'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:04.646180 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c716fad3-462e-41cc-85c3-5680c1854f90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a11ad72b5e845faacaea4d2361a1b9c', 'request_id': 'req-c716fad3-462e-41cc-85c3-5680c1854f90'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:04.652170 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c716fad3-462e-41cc-85c3-5680c1854f90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a11ad72b5e845faacaea4d2361a1b9c', 'request_id': 'req-c716fad3-462e-41cc-85c3-5680c1854f90'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:04.664556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c716fad3-462e-41cc-85c3-5680c1854f90 admin admin] The token cache is being invalidate because project 7a11ad72b5e845faacaea4d2361a1b9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:38:04.664841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c716fad3-462e-41cc-85c3-5680c1854f90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:04.665448 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c716fad3-462e-41cc-85c3-5680c1854f90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:04.666940 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1374/2735] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:38:04 2022] DELETE /identity/v3/projects/7a11ad72b5e845faacaea4d2361a1b9c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:04.671716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e2b1b-529b-4072-bc5a-70b3a5d3770b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:04.671995 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e2b1b-529b-4072-bc5a-70b3a5d3770b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:04.672265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e2b1b-529b-4072-bc5a-70b3a5d3770b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:04.691106 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-aa8e2b1b-529b-4072-bc5a-70b3a5d3770b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:04.692265 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-aa8e2b1b-529b-4072-bc5a-70b3a5d3770b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:04.767168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1362/2736] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:04.775064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19494e94-41db-4c0f-8096-e4f5451a7e21 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:04.775441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19494e94-41db-4c0f-8096-e4f5451a7e21 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:04.775796 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19494e94-41db-4c0f-8096-e4f5451a7e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:04.796060 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-19494e94-41db-4c0f-8096-e4f5451a7e21 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:04.797256 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-19494e94-41db-4c0f-8096-e4f5451a7e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:04.805108 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1375/2737] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:04.809361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e2b1b-529b-4072-bc5a-70b3a5d3770b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:04.818935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e2b1b-529b-4072-bc5a-70b3a5d3770b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:04.823475 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37150adf-cf44-473f-815c-b6dfb44c1a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:04.824054 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37150adf-cf44-473f-815c-b6dfb44c1a1a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:04.824329 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37150adf-cf44-473f-815c-b6dfb44c1a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:04.824599 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37150adf-cf44-473f-815c-b6dfb44c1a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:04.825240 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37150adf-cf44-473f-815c-b6dfb44c1a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:38:04.825536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37150adf-cf44-473f-815c-b6dfb44c1a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:04.827145 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37150adf-cf44-473f-815c-b6dfb44c1a1a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:04.839089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1363/2738] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 27 17:38:04.843391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19494e94-41db-4c0f-8096-e4f5451a7e21 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:04.853157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19494e94-41db-4c0f-8096-e4f5451a7e21 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:04.857852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85045aa-ea1a-4f0a-a612-da792f7f01c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:04.858481 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85045aa-ea1a-4f0a-a612-da792f7f01c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:04.858758 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85045aa-ea1a-4f0a-a612-da792f7f01c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:04.859026 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85045aa-ea1a-4f0a-a612-da792f7f01c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:04.859649 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85045aa-ea1a-4f0a-a612-da792f7f01c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:04.861810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85045aa-ea1a-4f0a-a612-da792f7f01c3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:04.888213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1376/2739] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:38:04 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:38:04.892749 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37150adf-cf44-473f-815c-b6dfb44c1a1a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:04.902596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37150adf-cf44-473f-815c-b6dfb44c1a1a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:04.914642 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:04.960766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c62b1-1dc7-4e34-af6c-85953993fa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:04.961399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c62b1-1dc7-4e34-af6c-85953993fa11 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:04.961623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c62b1-1dc7-4e34-af6c-85953993fa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:04.961894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c62b1-1dc7-4e34-af6c-85953993fa11 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:04.962549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c62b1-1dc7-4e34-af6c-85953993fa11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:04.964465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c62b1-1dc7-4e34-af6c-85953993fa11 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:04.985502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1364/2740] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:38:04 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:38:04.989867 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85045aa-ea1a-4f0a-a612-da792f7f01c3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:05.000178 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85045aa-ea1a-4f0a-a612-da792f7f01c3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:05.005253 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065b84e-8007-41d0-93a2-3e1a9359251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:05.005824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065b84e-8007-41d0-93a2-3e1a9359251a admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:05.006127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065b84e-8007-41d0-93a2-3e1a9359251a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:05.006403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065b84e-8007-41d0-93a2-3e1a9359251a admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:05.007073 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065b84e-8007-41d0-93a2-3e1a9359251a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:05.008896 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065b84e-8007-41d0-93a2-3e1a9359251a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:05.020760 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1377/2741] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:38:05.023597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c62b1-1dc7-4e34-af6c-85953993fa11 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:05.033314 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c62b1-1dc7-4e34-af6c-85953993fa11 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:05.037894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08485a-f95c-4cbd-ade3-a06cafac8240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:05.038524 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08485a-f95c-4cbd-ade3-a06cafac8240 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:05.038797 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08485a-f95c-4cbd-ade3-a06cafac8240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:05.039065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08485a-f95c-4cbd-ade3-a06cafac8240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:05.039697 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08485a-f95c-4cbd-ade3-a06cafac8240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:05.041282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a08485a-f95c-4cbd-ade3-a06cafac8240 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:05.051593 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1365/2742] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:38:05.055712 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065b84e-8007-41d0-93a2-3e1a9359251a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:05.066771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065b84e-8007-41d0-93a2-3e1a9359251a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:05.077104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464e4f5-b8d4-4119-aa9f-1f5f187dce89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:05.077757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464e4f5-b8d4-4119-aa9f-1f5f187dce89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:05.078031 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464e4f5-b8d4-4119-aa9f-1f5f187dce89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:05.078437 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464e4f5-b8d4-4119-aa9f-1f5f187dce89 admin admin] PATH_INFO: `/v3/projects/4dfaf9be57f547d9a19289cfd6e5125d/users/5a6ac10ea08443268ad841d1a077a898/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:05.093902 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464e4f5-b8d4-4119-aa9f-1f5f187dce89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dfaf9be57f547d9a19289cfd6e5125d, user_id=5a6ac10ea08443268ad841d1a077a898, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:05.096715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464e4f5-b8d4-4119-aa9f-1f5f187dce89 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:05.111837 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1378/2743] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:38:05 2022] PUT /identity/v3/projects/4dfaf9be57f547d9a19289cfd6e5125d/users/5a6ac10ea08443268ad841d1a077a898/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:05.175385 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ba264-d35c-4ea2-9146-eebd392b0d2f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:05.175664 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ba264-d35c-4ea2-9146-eebd392b0d2f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:05.175931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ba264-d35c-4ea2-9146-eebd392b0d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:05.205348 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ae8ba264-d35c-4ea2-9146-eebd392b0d2f None None] MFA Rules not processed for user `5a6ac10ea08443268ad841d1a077a898`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:05.206595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ae8ba264-d35c-4ea2-9146-eebd392b0d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:05.268675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1366/2744] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:05.279624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464e4f5-b8d4-4119-aa9f-1f5f187dce89 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:05.281310 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9464e4f5-b8d4-4119-aa9f-1f5f187dce89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:05.337687 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464e4f5-b8d4-4119-aa9f-1f5f187dce89 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:05.341374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16058f44-7c89-434e-96b1-fabe4a1377d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:05.341972 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16058f44-7c89-434e-96b1-fabe4a1377d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:05.342279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16058f44-7c89-434e-96b1-fabe4a1377d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:05.342550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16058f44-7c89-434e-96b1-fabe4a1377d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:05.350793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16058f44-7c89-434e-96b1-fabe4a1377d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:05.352236 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16058f44-7c89-434e-96b1-fabe4a1377d7 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:05.361908 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1379/2745] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:38:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:38:05.386584 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ba264-d35c-4ea2-9146-eebd392b0d2f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:05.456555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ba264-d35c-4ea2-9146-eebd392b0d2f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:05.461053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:05.461660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:05.461931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:05.462231 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] PATH_INFO: `/v3/users/5a6ac10ea08443268ad841d1a077a898` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:05.463339 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a6ac10ea08443268ad841d1a077a898)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:05.465224 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:05.491652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:05.510603 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a6ac10ea08443268ad841d1a077a898'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:05.511284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a6ac10ea08443268ad841d1a077a898'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:05.517953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a6ac10ea08443268ad841d1a077a898'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:05.524850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a6ac10ea08443268ad841d1a077a898'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:05.536058 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a6ac10ea08443268ad841d1a077a898'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:05.541683 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1367/2746] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:38:05 2022] DELETE /identity/v3/users/5a6ac10ea08443268ad841d1a077a898 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:05.546121 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda481b-c266-46a4-a102-c61d2804315a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:05.546595 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda481b-c266-46a4-a102-c61d2804315a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:05.546710 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda481b-c266-46a4-a102-c61d2804315a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:05.565855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7bda481b-c266-46a4-a102-c61d2804315a None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:05.567066 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7bda481b-c266-46a4-a102-c61d2804315a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:05.605687 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:05.624370 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1380/2747] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:05.634443 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:05.635835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:05.696120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f9156-00e4-4703-bcb9-e1d07e4e3335 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:05.701788 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5dd81-d1f1-49af-8d3b-4b30070f78aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:05.702408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5dd81-d1f1-49af-8d3b-4b30070f78aa service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:05.702688 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5dd81-d1f1-49af-8d3b-4b30070f78aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:05.702955 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5dd81-d1f1-49af-8d3b-4b30070f78aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:05.713068 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5dd81-d1f1-49af-8d3b-4b30070f78aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:05.714533 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5dd81-d1f1-49af-8d3b-4b30070f78aa service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:05.724293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1368/2748] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:38:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:38:05.980570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda481b-c266-46a4-a102-c61d2804315a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:05.982008 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7bda481b-c266-46a4-a102-c61d2804315a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:05.994602 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda481b-c266-46a4-a102-c61d2804315a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:05.999168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:05.999792 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.000066 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.000333 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7 admin admin] PATH_INFO: `/v3/projects/4dfaf9be57f547d9a19289cfd6e5125d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.001449 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dfaf9be57f547d9a19289cfd6e5125d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:06.003375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:06.049906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dfaf9be57f547d9a19289cfd6e5125d', 'request_id': 'req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:06.055379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dfaf9be57f547d9a19289cfd6e5125d', 'request_id': 'req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:06.061460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dfaf9be57f547d9a19289cfd6e5125d', 'request_id': 'req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:06.073570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7 admin admin] The token cache is being invalidate because project 4dfaf9be57f547d9a19289cfd6e5125d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:38:06.073798 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:06.074432 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-312c4fa6-bdac-435f-b2c1-b13cb0c6ebe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:06.075901 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1381/2749] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:38:05 2022] DELETE /identity/v3/projects/4dfaf9be57f547d9a19289cfd6e5125d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:06.081225 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08085-cec4-433f-8a62-3b4ad5af8a70 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.081689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08085-cec4-433f-8a62-3b4ad5af8a70 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.081784 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08085-cec4-433f-8a62-3b4ad5af8a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.102540 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-cfe08085-cec4-433f-8a62-3b4ad5af8a70 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:06.103739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cfe08085-cec4-433f-8a62-3b4ad5af8a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:06.162411 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1369/2750] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:06.169928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff214d0a-15fa-40be-af40-8c4c4ff1a07b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.170240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff214d0a-15fa-40be-af40-8c4c4ff1a07b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.170515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff214d0a-15fa-40be-af40-8c4c4ff1a07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.189876 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ff214d0a-15fa-40be-af40-8c4c4ff1a07b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:06.191115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ff214d0a-15fa-40be-af40-8c4c4ff1a07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:06.198937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1382/2751] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:06.202931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08085-cec4-433f-8a62-3b4ad5af8a70 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:06.212781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08085-cec4-433f-8a62-3b4ad5af8a70 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:06.217278 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8a1b1-6b3a-40c9-ad47-0077f5b41730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:06.217859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8a1b1-6b3a-40c9-ad47-0077f5b41730 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.218171 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8a1b1-6b3a-40c9-ad47-0077f5b41730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.218442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8a1b1-6b3a-40c9-ad47-0077f5b41730 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.219675 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8a1b1-6b3a-40c9-ad47-0077f5b41730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:38:06.219983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8a1b1-6b3a-40c9-ad47-0077f5b41730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:06.221547 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8a1b1-6b3a-40c9-ad47-0077f5b41730 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:06.232739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1370/2752] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:38:06.237244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff214d0a-15fa-40be-af40-8c4c4ff1a07b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:06.246455 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff214d0a-15fa-40be-af40-8c4c4ff1a07b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:06.251052 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e0227-9619-4eef-b51d-3260c4ee61d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:06.251632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e0227-9619-4eef-b51d-3260c4ee61d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.251903 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e0227-9619-4eef-b51d-3260c4ee61d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.252169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e0227-9619-4eef-b51d-3260c4ee61d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.252779 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e0227-9619-4eef-b51d-3260c4ee61d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:06.254684 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e0227-9619-4eef-b51d-3260c4ee61d9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:06.282937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1383/2753] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:38:06 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:38:06.287075 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8a1b1-6b3a-40c9-ad47-0077f5b41730 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:06.296793 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8a1b1-6b3a-40c9-ad47-0077f5b41730 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:06.327263 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:06.356373 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b614b-4ce6-4019-9871-e59c6fee3e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:06.356972 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b614b-4ce6-4019-9871-e59c6fee3e09 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.357249 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b614b-4ce6-4019-9871-e59c6fee3e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.357582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b614b-4ce6-4019-9871-e59c6fee3e09 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.358234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b614b-4ce6-4019-9871-e59c6fee3e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:06.360141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b614b-4ce6-4019-9871-e59c6fee3e09 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:06.380660 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1371/2754] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:38:06 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:38:06.384728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e0227-9619-4eef-b51d-3260c4ee61d9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:06.395361 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e0227-9619-4eef-b51d-3260c4ee61d9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:06.399962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba351eb-8915-44b0-8d29-c8614962c6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:06.400548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba351eb-8915-44b0-8d29-c8614962c6cb admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.400826 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba351eb-8915-44b0-8d29-c8614962c6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.401098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba351eb-8915-44b0-8d29-c8614962c6cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.401737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba351eb-8915-44b0-8d29-c8614962c6cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:06.403548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba351eb-8915-44b0-8d29-c8614962c6cb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:06.414387 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1384/2755] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:38:06.418297 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b614b-4ce6-4019-9871-e59c6fee3e09 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:06.427961 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b614b-4ce6-4019-9871-e59c6fee3e09 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:06.432505 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1e8e8-06d6-4cfb-a84d-09445baac4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:06.433083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1e8e8-06d6-4cfb-a84d-09445baac4ce admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.433358 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1e8e8-06d6-4cfb-a84d-09445baac4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.433628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1e8e8-06d6-4cfb-a84d-09445baac4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.434301 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1e8e8-06d6-4cfb-a84d-09445baac4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:06.435864 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1e8e8-06d6-4cfb-a84d-09445baac4ce admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:06.446634 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1372/2756] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:38:06.450490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba351eb-8915-44b0-8d29-c8614962c6cb admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:06.460862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba351eb-8915-44b0-8d29-c8614962c6cb admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:06.465457 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f1993-f22b-44d6-b8eb-e84ad5625181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:06.466118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f1993-f22b-44d6-b8eb-e84ad5625181 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.466403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f1993-f22b-44d6-b8eb-e84ad5625181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.466676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f1993-f22b-44d6-b8eb-e84ad5625181 admin admin] PATH_INFO: `/v3/projects/666ce592963f4cd38853da4bc661b204/users/63f730ebc9534a72b50d2da19563c954/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.480834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f1993-f22b-44d6-b8eb-e84ad5625181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=666ce592963f4cd38853da4bc661b204, user_id=63f730ebc9534a72b50d2da19563c954, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:06.483521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f1993-f22b-44d6-b8eb-e84ad5625181 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:06.496909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1385/2757] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:38:06 2022] PUT /identity/v3/projects/666ce592963f4cd38853da4bc661b204/users/63f730ebc9534a72b50d2da19563c954/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:06.507759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b0f6f-ac69-4ccd-a37e-49f332c80183 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.508042 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b0f6f-ac69-4ccd-a37e-49f332c80183 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.508316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b0f6f-ac69-4ccd-a37e-49f332c80183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.535666 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-111b0f6f-ac69-4ccd-a37e-49f332c80183 None None] MFA Rules not processed for user `63f730ebc9534a72b50d2da19563c954`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:06.536864 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-111b0f6f-ac69-4ccd-a37e-49f332c80183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:06.598538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1373/2758] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:06.603113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f1993-f22b-44d6-b8eb-e84ad5625181 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:06.663624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f1993-f22b-44d6-b8eb-e84ad5625181 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:06.668240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e109d3f-8e8c-4b15-bc1a-3481be73bed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:06.668835 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e109d3f-8e8c-4b15-bc1a-3481be73bed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.669112 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e109d3f-8e8c-4b15-bc1a-3481be73bed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.669385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e109d3f-8e8c-4b15-bc1a-3481be73bed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.670009 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e109d3f-8e8c-4b15-bc1a-3481be73bed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:06.671962 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e109d3f-8e8c-4b15-bc1a-3481be73bed8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:06.682659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:06.701151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1386/2759] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:38:06 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:38:06.705811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b0f6f-ac69-4ccd-a37e-49f332c80183 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:06.716302 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b0f6f-ac69-4ccd-a37e-49f332c80183 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:06.772968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77375e96-5596-4f5d-b798-08be00f692e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:06.773572 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77375e96-5596-4f5d-b798-08be00f692e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.773845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77375e96-5596-4f5d-b798-08be00f692e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.774132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77375e96-5596-4f5d-b798-08be00f692e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.774764 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77375e96-5596-4f5d-b798-08be00f692e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:06.776658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77375e96-5596-4f5d-b798-08be00f692e2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:06.800113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1374/2760] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:38:06 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:38:06.804017 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e109d3f-8e8c-4b15-bc1a-3481be73bed8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:06.815093 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e109d3f-8e8c-4b15-bc1a-3481be73bed8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:06.819672 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2894fa0-9c22-4974-8ddc-b353bd3654f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:06.820267 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2894fa0-9c22-4974-8ddc-b353bd3654f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.820545 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2894fa0-9c22-4974-8ddc-b353bd3654f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.820816 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2894fa0-9c22-4974-8ddc-b353bd3654f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.821494 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2894fa0-9c22-4974-8ddc-b353bd3654f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:06.823106 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2894fa0-9c22-4974-8ddc-b353bd3654f0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:06.832824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1387/2761] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:38:06.836574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77375e96-5596-4f5d-b798-08be00f692e2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:06.846497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77375e96-5596-4f5d-b798-08be00f692e2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:06.851229 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a6d8a-2b7d-4db3-973b-c2c54486e17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:06.851781 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a6d8a-2b7d-4db3-973b-c2c54486e17b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.852053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a6d8a-2b7d-4db3-973b-c2c54486e17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.852322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a6d8a-2b7d-4db3-973b-c2c54486e17b admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.852958 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a6d8a-2b7d-4db3-973b-c2c54486e17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:06.854616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a6d8a-2b7d-4db3-973b-c2c54486e17b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:06.865152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1375/2762] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:38:06.869071 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2894fa0-9c22-4974-8ddc-b353bd3654f0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:06.879523 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2894fa0-9c22-4974-8ddc-b353bd3654f0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:06.884181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0883754-8eb9-4f26-ae35-0ee0f6335227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:06.884811 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0883754-8eb9-4f26-ae35-0ee0f6335227 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.885088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0883754-8eb9-4f26-ae35-0ee0f6335227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.885359 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0883754-8eb9-4f26-ae35-0ee0f6335227 admin admin] PATH_INFO: `/v3/projects/1b741b8546fb435393666858aaade5e1/users/551a8e12ef34483eb50960f3f466c18f/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.899533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0883754-8eb9-4f26-ae35-0ee0f6335227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b741b8546fb435393666858aaade5e1, user_id=551a8e12ef34483eb50960f3f466c18f, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:06.902325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0883754-8eb9-4f26-ae35-0ee0f6335227 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:06.915130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1388/2763] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:38:06 2022] PUT /identity/v3/projects/1b741b8546fb435393666858aaade5e1/users/551a8e12ef34483eb50960f3f466c18f/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:06.926775 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff28c88-6ecf-4021-8635-1c5b9d19e191 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:06.927060 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff28c88-6ecf-4021-8635-1c5b9d19e191 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:06.927334 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff28c88-6ecf-4021-8635-1c5b9d19e191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:06.955722 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6ff28c88-6ecf-4021-8635-1c5b9d19e191 None None] MFA Rules not processed for user `551a8e12ef34483eb50960f3f466c18f`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:06.956902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6ff28c88-6ecf-4021-8635-1c5b9d19e191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:06.964733 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:07.020232 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1376/2764] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:07.024806 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0883754-8eb9-4f26-ae35-0ee0f6335227 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:07.085078 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0883754-8eb9-4f26-ae35-0ee0f6335227 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:07.089618 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ccd11-883a-49ef-bbb4-96678049d6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:07.090231 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ccd11-883a-49ef-bbb4-96678049d6bc admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:07.090506 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ccd11-883a-49ef-bbb4-96678049d6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:07.090770 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ccd11-883a-49ef-bbb4-96678049d6bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:07.091398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ccd11-883a-49ef-bbb4-96678049d6bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:07.093287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ccd11-883a-49ef-bbb4-96678049d6bc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:07.119225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1389/2765] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:38:07 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:38:07.123262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff28c88-6ecf-4021-8635-1c5b9d19e191 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:07.134393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff28c88-6ecf-4021-8635-1c5b9d19e191 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:07.190707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa26dd-0380-452a-9e1d-3fcfcede7879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:07.191306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa26dd-0380-452a-9e1d-3fcfcede7879 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:07.191586 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa26dd-0380-452a-9e1d-3fcfcede7879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:07.191856 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fa26dd-0380-452a-9e1d-3fcfcede7879 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:07.192477 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa26dd-0380-452a-9e1d-3fcfcede7879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:07.194407 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fa26dd-0380-452a-9e1d-3fcfcede7879 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:07.214802 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1377/2766] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:38:07 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:38:07.218832 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ccd11-883a-49ef-bbb4-96678049d6bc admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:07.229402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ccd11-883a-49ef-bbb4-96678049d6bc admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:07.233932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bc04e-e037-4015-9349-df54fade735d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:07.234548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc04e-e037-4015-9349-df54fade735d admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:07.235390 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc04e-e037-4015-9349-df54fade735d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:07.235676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bc04e-e037-4015-9349-df54fade735d admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:07.236572 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bc04e-e037-4015-9349-df54fade735d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:07.238193 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bc04e-e037-4015-9349-df54fade735d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:07.248608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1390/2767] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:38:07.252355 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa26dd-0380-452a-9e1d-3fcfcede7879 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:07.262031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fa26dd-0380-452a-9e1d-3fcfcede7879 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:07.266647 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c338e-6f89-4b78-ab65-529198329b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:07.267271 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c338e-6f89-4b78-ab65-529198329b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:07.268136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c338e-6f89-4b78-ab65-529198329b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:07.268136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c338e-6f89-4b78-ab65-529198329b6a admin admin] PATH_INFO: `/v3/projects/31fc04ed97a64a24917ac8d3fd14ada7/users/2e695235178e4a80a5cdace0c7905631/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:07.282595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c338e-6f89-4b78-ab65-529198329b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=31fc04ed97a64a24917ac8d3fd14ada7, user_id=2e695235178e4a80a5cdace0c7905631, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:07.285221 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c338e-6f89-4b78-ab65-529198329b6a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:07.298746 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1378/2768] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:38:07 2022] PUT /identity/v3/projects/31fc04ed97a64a24917ac8d3fd14ada7/users/2e695235178e4a80a5cdace0c7905631/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:07.302788 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bc04e-e037-4015-9349-df54fade735d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:07.312852 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bc04e-e037-4015-9349-df54fade735d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:07.370946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed2535-31af-4771-9b56-7bfccb1bcde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:07.371484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed2535-31af-4771-9b56-7bfccb1bcde6 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:07.371764 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed2535-31af-4771-9b56-7bfccb1bcde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:07.372036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed2535-31af-4771-9b56-7bfccb1bcde6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:07.372679 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed2535-31af-4771-9b56-7bfccb1bcde6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:07.374307 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed2535-31af-4771-9b56-7bfccb1bcde6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:07.383849 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1391/2769] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:38:07.388152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c338e-6f89-4b78-ab65-529198329b6a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:07.398352 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c338e-6f89-4b78-ab65-529198329b6a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:07.402884 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c7023-fb7b-4838-9113-24f3cb3707bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:07.403511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c7023-fb7b-4838-9113-24f3cb3707bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:07.403785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c7023-fb7b-4838-9113-24f3cb3707bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:07.404053 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c7023-fb7b-4838-9113-24f3cb3707bf admin admin] PATH_INFO: `/v3/projects/31fc04ed97a64a24917ac8d3fd14ada7/users/2e695235178e4a80a5cdace0c7905631/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:07.406042 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c7023-fb7b-4838-9113-24f3cb3707bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=31fc04ed97a64a24917ac8d3fd14ada7, user_id=2e695235178e4a80a5cdace0c7905631, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:07.409160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498c7023-fb7b-4838-9113-24f3cb3707bf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:07.419682 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1379/2770] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:38:07 2022] PUT /identity/v3/projects/31fc04ed97a64a24917ac8d3fd14ada7/users/2e695235178e4a80a5cdace0c7905631/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:07.430152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ed5d3-9ecc-465b-90fa-a95111dfea47 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:07.430523 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ed5d3-9ecc-465b-90fa-a95111dfea47 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:07.430859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ed5d3-9ecc-465b-90fa-a95111dfea47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:07.457615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ef2ed5d3-9ecc-465b-90fa-a95111dfea47 None None] MFA Rules not processed for user `2e695235178e4a80a5cdace0c7905631`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:07.458841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ef2ed5d3-9ecc-465b-90fa-a95111dfea47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:07.471700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:07.535042 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1392/2771] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:07.546247 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c7023-fb7b-4838-9113-24f3cb3707bf admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:07.548612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-498c7023-fb7b-4838-9113-24f3cb3707bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:07.605027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c7023-fb7b-4838-9113-24f3cb3707bf admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:07.608889 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edea71-40e1-4a06-b175-b1ab777243b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:07.609543 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edea71-40e1-4a06-b175-b1ab777243b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:07.609850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edea71-40e1-4a06-b175-b1ab777243b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:07.610149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edea71-40e1-4a06-b175-b1ab777243b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:07.622862 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:07.666467 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edea71-40e1-4a06-b175-b1ab777243b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:07.667960 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edea71-40e1-4a06-b175-b1ab777243b9 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:07.676583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1380/2772] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:38:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:38:09.182147 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ed5d3-9ecc-465b-90fa-a95111dfea47 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:09.183587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ef2ed5d3-9ecc-465b-90fa-a95111dfea47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:09.195381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ed5d3-9ecc-465b-90fa-a95111dfea47 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:09.199206 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e5fba-3f03-4758-9e08-0ef05b003799 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:09.199927 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e5fba-3f03-4758-9e08-0ef05b003799 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:09.200209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e5fba-3f03-4758-9e08-0ef05b003799 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:09.200478 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e5fba-3f03-4758-9e08-0ef05b003799 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:09.256102 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e5fba-3f03-4758-9e08-0ef05b003799 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:09.257524 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e5fba-3f03-4758-9e08-0ef05b003799 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:09.266612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1393/2773] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:38:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:38:09.788280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edea71-40e1-4a06-b175-b1ab777243b9 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:09.796919 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edea71-40e1-4a06-b175-b1ab777243b9 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:09.801062 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d5501-6a58-4d6c-b463-f912d118d8e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:09.801654 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d5501-6a58-4d6c-b463-f912d118d8e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:09.801934 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d5501-6a58-4d6c-b463-f912d118d8e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:09.802237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d5501-6a58-4d6c-b463-f912d118d8e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:09.811954 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d5501-6a58-4d6c-b463-f912d118d8e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:09.814174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d5501-6a58-4d6c-b463-f912d118d8e6 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:09.823528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1381/2774] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:38:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:38:13.679915 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e5fba-3f03-4758-9e08-0ef05b003799 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:13.740946 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e5fba-3f03-4758-9e08-0ef05b003799 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:13.745568 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:13.746213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:13.746493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:13.746761 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] PATH_INFO: `/v3/users/63f730ebc9534a72b50d2da19563c954` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:13.747877 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=63f730ebc9534a72b50d2da19563c954)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:13.749783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:13.798172 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63f730ebc9534a72b50d2da19563c954'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:13.802451 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:13.805591 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63f730ebc9534a72b50d2da19563c954'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:13.806741 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63f730ebc9534a72b50d2da19563c954'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:13.815155 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63f730ebc9534a72b50d2da19563c954'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:13.826863 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63f730ebc9534a72b50d2da19563c954'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:13.832587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1394/2775] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:38:13 2022] DELETE /identity/v3/users/63f730ebc9534a72b50d2da19563c954 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:13.837036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18422194-14c5-48cd-b66a-d1b508195e9b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:13.837528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18422194-14c5-48cd-b66a-d1b508195e9b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:13.837638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18422194-14c5-48cd-b66a-d1b508195e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:13.855777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-18422194-14c5-48cd-b66a-d1b508195e9b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:13.856950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-18422194-14c5-48cd-b66a-d1b508195e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:13.913031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1382/2776] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:13.923117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:13.924493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:13.981034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272bca0-ca58-48a1-9fe3-d65d937c2d6d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:13.985270 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff50d1-3d0c-4621-b2a0-386ad75e66d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:13.985844 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff50d1-3d0c-4621-b2a0-386ad75e66d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:13.986158 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff50d1-3d0c-4621-b2a0-386ad75e66d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:13.986694 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff50d1-3d0c-4621-b2a0-386ad75e66d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:13.995456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff50d1-3d0c-4621-b2a0-386ad75e66d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:13.996869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff50d1-3d0c-4621-b2a0-386ad75e66d6 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:14.006123 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1395/2777] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:38:14.267578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18422194-14c5-48cd-b66a-d1b508195e9b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:14.269136 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-18422194-14c5-48cd-b66a-d1b508195e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:14.281069 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18422194-14c5-48cd-b66a-d1b508195e9b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:14.286174 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:14.286831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:14.287105 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:14.287375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] PATH_INFO: `/v3/projects/666ce592963f4cd38853da4bc661b204` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:14.288514 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] RBAC: Authorizing `identity:delete_project(project_id=666ce592963f4cd38853da4bc661b204)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:14.290452 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:14.337509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '666ce592963f4cd38853da4bc661b204', 'request_id': 'req-d6b9875e-cbf9-4771-99c2-b189efaedfda'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.342905 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '666ce592963f4cd38853da4bc661b204', 'request_id': 'req-d6b9875e-cbf9-4771-99c2-b189efaedfda'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.349629 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '666ce592963f4cd38853da4bc661b204', 'request_id': 'req-d6b9875e-cbf9-4771-99c2-b189efaedfda'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.361217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] The token cache is being invalidate because project 666ce592963f4cd38853da4bc661b204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:38:14.361752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.362377 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.363829 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1383/2778] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:38:14 2022] DELETE /identity/v3/projects/666ce592963f4cd38853da4bc661b204 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:14.367836 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff50d1-3d0c-4621-b2a0-386ad75e66d6 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:14.377096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff50d1-3d0c-4621-b2a0-386ad75e66d6 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:14.431297 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:14.431904 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:14.432181 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:14.432482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] PATH_INFO: `/v3/users/551a8e12ef34483eb50960f3f466c18f` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:14.433550 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=551a8e12ef34483eb50960f3f466c18f)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:14.435481 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:14.482056 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '551a8e12ef34483eb50960f3f466c18f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.490145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '551a8e12ef34483eb50960f3f466c18f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.490810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '551a8e12ef34483eb50960f3f466c18f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.498805 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '551a8e12ef34483eb50960f3f466c18f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.507021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:14.515461 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '551a8e12ef34483eb50960f3f466c18f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.524164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1396/2779] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:38:14 2022] DELETE /identity/v3/users/551a8e12ef34483eb50960f3f466c18f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:38:14.786089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:14.787617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:14.800478 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9875e-cbf9-4771-99c2-b189efaedfda admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:14.824979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:14.858381 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:14.859020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:14.859274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:14.859550 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] PATH_INFO: `/v3/projects/1b741b8546fb435393666858aaade5e1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:14.860665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b741b8546fb435393666858aaade5e1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:14.862863 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:14.909254 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b741b8546fb435393666858aaade5e1', 'request_id': 'req-a17627e6-2907-4b17-bab6-9128020c2162'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.914751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b741b8546fb435393666858aaade5e1', 'request_id': 'req-a17627e6-2907-4b17-bab6-9128020c2162'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.921030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b741b8546fb435393666858aaade5e1', 'request_id': 'req-a17627e6-2907-4b17-bab6-9128020c2162'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.932668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] The token cache is being invalidate because project 1b741b8546fb435393666858aaade5e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:38:14.932953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.933574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:14.935056 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1384/2780] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:38:14 2022] DELETE /identity/v3/projects/1b741b8546fb435393666858aaade5e1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:14.939662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:14.941036 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:14.954040 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb8893-585e-47f2-9213-f32e949082ba admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:15.007808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabc5fc-8e4e-4a39-a19c-4facc24bce10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:15.008354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabc5fc-8e4e-4a39-a19c-4facc24bce10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:15.008625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabc5fc-8e4e-4a39-a19c-4facc24bce10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:15.008890 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabc5fc-8e4e-4a39-a19c-4facc24bce10 admin admin] PATH_INFO: `/v3/users/2e695235178e4a80a5cdace0c7905631` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:15.009971 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabc5fc-8e4e-4a39-a19c-4facc24bce10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e695235178e4a80a5cdace0c7905631)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:15.011963 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabc5fc-8e4e-4a39-a19c-4facc24bce10 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:15.056918 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fdabc5fc-8e4e-4a39-a19c-4facc24bce10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e695235178e4a80a5cdace0c7905631'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:15.064147 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fdabc5fc-8e4e-4a39-a19c-4facc24bce10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e695235178e4a80a5cdace0c7905631'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:15.064787 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fdabc5fc-8e4e-4a39-a19c-4facc24bce10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e695235178e4a80a5cdace0c7905631'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:15.072303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fdabc5fc-8e4e-4a39-a19c-4facc24bce10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e695235178e4a80a5cdace0c7905631'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:15.083854 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fdabc5fc-8e4e-4a39-a19c-4facc24bce10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e695235178e4a80a5cdace0c7905631'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:15.089615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1397/2781] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:38:14 2022] DELETE /identity/v3/users/2e695235178e4a80a5cdace0c7905631 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:38:15.357502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:15.359001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:15.372773 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17627e6-2907-4b17-bab6-9128020c2162 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:15.432354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:15.432987 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:15.433261 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:15.433532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3 admin admin] PATH_INFO: `/v3/projects/31fc04ed97a64a24917ac8d3fd14ada7` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:15.434670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31fc04ed97a64a24917ac8d3fd14ada7)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:15.436578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:15.486904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31fc04ed97a64a24917ac8d3fd14ada7', 'request_id': 'req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:15.493346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31fc04ed97a64a24917ac8d3fd14ada7', 'request_id': 'req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:15.494439 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:15.501326 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31fc04ed97a64a24917ac8d3fd14ada7', 'request_id': 'req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:15.513707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3 admin admin] The token cache is being invalidate because project 31fc04ed97a64a24917ac8d3fd14ada7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:38:15.513990 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:15.514619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7aed0efb-9b9b-4870-b183-7f1d3fd790a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:38:15.516089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1385/2782] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:38:15 2022] DELETE /identity/v3/projects/31fc04ed97a64a24917ac8d3fd14ada7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:15.521183 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d28e7-85d9-4244-b1c6-7339662bfbe4 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:15.521455 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d28e7-85d9-4244-b1c6-7339662bfbe4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:15.521749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d28e7-85d9-4244-b1c6-7339662bfbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:15.541258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e05d28e7-85d9-4244-b1c6-7339662bfbe4 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:15.542476 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e05d28e7-85d9-4244-b1c6-7339662bfbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:15.601313 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1398/2783] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:15.611076 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e647838-d77d-48a3-9360-06fb7c96cfe5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:15.611629 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e647838-d77d-48a3-9360-06fb7c96cfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:15.611900 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e647838-d77d-48a3-9360-06fb7c96cfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:15.631197 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1e647838-d77d-48a3-9360-06fb7c96cfe5 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:15.632398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1e647838-d77d-48a3-9360-06fb7c96cfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:15.640277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1386/2784] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:15.644506 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d28e7-85d9-4244-b1c6-7339662bfbe4 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:15.654121 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d28e7-85d9-4244-b1c6-7339662bfbe4 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:15.659185 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f1d25-80b4-4a4b-999f-6f15216d617a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:15.659769 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f1d25-80b4-4a4b-999f-6f15216d617a admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:15.660047 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f1d25-80b4-4a4b-999f-6f15216d617a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:15.660316 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f1d25-80b4-4a4b-999f-6f15216d617a admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:15.660959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f1d25-80b4-4a4b-999f-6f15216d617a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:38:15.661260 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f1d25-80b4-4a4b-999f-6f15216d617a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:15.662886 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f1d25-80b4-4a4b-999f-6f15216d617a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:15.669404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:15.676435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1399/2785] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:38:15.679930 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e647838-d77d-48a3-9360-06fb7c96cfe5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:15.689075 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e647838-d77d-48a3-9360-06fb7c96cfe5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:15.693678 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604eb8e-3234-4cd8-916f-acf3d5f04b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:15.694283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604eb8e-3234-4cd8-916f-acf3d5f04b9e admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:15.694561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604eb8e-3234-4cd8-916f-acf3d5f04b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:15.694827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604eb8e-3234-4cd8-916f-acf3d5f04b9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:15.695452 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604eb8e-3234-4cd8-916f-acf3d5f04b9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:15.697330 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604eb8e-3234-4cd8-916f-acf3d5f04b9e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:15.723967 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1387/2786] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:38:15 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:38:15.727900 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f1d25-80b4-4a4b-999f-6f15216d617a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:15.737983 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f1d25-80b4-4a4b-999f-6f15216d617a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:15.792978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d08bb8-14b6-418c-99ad-43d6857941c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:15.793496 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d08bb8-14b6-418c-99ad-43d6857941c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:15.793791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d08bb8-14b6-418c-99ad-43d6857941c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:15.794081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d08bb8-14b6-418c-99ad-43d6857941c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:15.794693 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d08bb8-14b6-418c-99ad-43d6857941c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:15.796647 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d08bb8-14b6-418c-99ad-43d6857941c5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:15.818652 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1400/2787] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:38:15 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:38:15.822880 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604eb8e-3234-4cd8-916f-acf3d5f04b9e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:15.833092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604eb8e-3234-4cd8-916f-acf3d5f04b9e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:15.837644 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b76ae-531a-48a0-8a91-dbcbbec544b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:15.838267 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b76ae-531a-48a0-8a91-dbcbbec544b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:15.838575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b76ae-531a-48a0-8a91-dbcbbec544b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:15.838846 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b76ae-531a-48a0-8a91-dbcbbec544b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:15.839508 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b76ae-531a-48a0-8a91-dbcbbec544b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:15.841083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b76ae-531a-48a0-8a91-dbcbbec544b2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:15.850784 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1388/2788] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:38:15.854620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d08bb8-14b6-418c-99ad-43d6857941c5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:15.865076 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d08bb8-14b6-418c-99ad-43d6857941c5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:15.869608 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af301c9-b66b-4303-a1b6-e5f20556e643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:15.870212 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af301c9-b66b-4303-a1b6-e5f20556e643 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:15.870485 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af301c9-b66b-4303-a1b6-e5f20556e643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:15.870749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af301c9-b66b-4303-a1b6-e5f20556e643 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:15.871375 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af301c9-b66b-4303-a1b6-e5f20556e643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:15.872942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af301c9-b66b-4303-a1b6-e5f20556e643 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:15.884132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1401/2789] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:38:15.888002 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b76ae-531a-48a0-8a91-dbcbbec544b2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:15.898040 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b76ae-531a-48a0-8a91-dbcbbec544b2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:15.902677 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efca396-18c1-4c65-9de4-133c09c8ce6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:15.903303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efca396-18c1-4c65-9de4-133c09c8ce6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:15.903579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efca396-18c1-4c65-9de4-133c09c8ce6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:15.903847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efca396-18c1-4c65-9de4-133c09c8ce6d admin admin] PATH_INFO: `/v3/projects/7a5cb427a38643a9b547e710d1c9caf5/users/53fb0b1bfcb6449089f3162f88145a14/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:15.917841 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efca396-18c1-4c65-9de4-133c09c8ce6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5cb427a38643a9b547e710d1c9caf5, user_id=53fb0b1bfcb6449089f3162f88145a14, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:15.920526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efca396-18c1-4c65-9de4-133c09c8ce6d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:15.933573 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1389/2790] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:38:15 2022] PUT /identity/v3/projects/7a5cb427a38643a9b547e710d1c9caf5/users/53fb0b1bfcb6449089f3162f88145a14/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:38:15.939251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac19bb-fc2b-4539-9a7d-896ad5e3e33e None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:15.939632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac19bb-fc2b-4539-9a7d-896ad5e3e33e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:15.939999 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac19bb-fc2b-4539-9a7d-896ad5e3e33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:15.959652 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d6ac19bb-fc2b-4539-9a7d-896ad5e3e33e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:15.960833 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d6ac19bb-fc2b-4539-9a7d-896ad5e3e33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:16.017146 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1402/2791] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:16.026695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efca396-18c1-4c65-9de4-133c09c8ce6d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:16.028382 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0efca396-18c1-4c65-9de4-133c09c8ce6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:16.086811 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efca396-18c1-4c65-9de4-133c09c8ce6d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:16.089331 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d38dc-6e31-4367-9483-516f421ad848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:16.089931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d38dc-6e31-4367-9483-516f421ad848 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:16.090233 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d38dc-6e31-4367-9483-516f421ad848 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:16.090502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694d38dc-6e31-4367-9483-516f421ad848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:16.103563 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d38dc-6e31-4367-9483-516f421ad848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:16.105002 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694d38dc-6e31-4367-9483-516f421ad848 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:16.115312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1390/2792] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:38:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:38:16.718834 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573bcff-a8f5-4451-ad6b-6680ddad4210 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:16.719752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573bcff-a8f5-4451-ad6b-6680ddad4210 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:16.720101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573bcff-a8f5-4451-ad6b-6680ddad4210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:16.748357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7573bcff-a8f5-4451-ad6b-6680ddad4210 None None] MFA Rules not processed for user `53fb0b1bfcb6449089f3162f88145a14`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:16.749570 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7573bcff-a8f5-4451-ad6b-6680ddad4210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:16.815954 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1403/2793] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:16.826848 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d38dc-6e31-4367-9483-516f421ad848 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:16.828860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-694d38dc-6e31-4367-9483-516f421ad848 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:16.890661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694d38dc-6e31-4367-9483-516f421ad848 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:16.895492 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da4d58-0669-42b8-8d21-7bd53824dfe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:16.896103 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da4d58-0669-42b8-8d21-7bd53824dfe9 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:16.896379 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da4d58-0669-42b8-8d21-7bd53824dfe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:16.899737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da4d58-0669-42b8-8d21-7bd53824dfe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:16.909123 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da4d58-0669-42b8-8d21-7bd53824dfe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:16.910585 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da4d58-0669-42b8-8d21-7bd53824dfe9 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:16.919480 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1391/2794] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:38:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:38:18.274765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459037e-bde9-4df8-8cf9-0d6818a3102a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:18.275254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459037e-bde9-4df8-8cf9-0d6818a3102a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:18.275378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459037e-bde9-4df8-8cf9-0d6818a3102a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:18.289467 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:18.295287 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d459037e-bde9-4df8-8cf9-0d6818a3102a None None] MFA Rules not processed for user `53fb0b1bfcb6449089f3162f88145a14`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:18.296472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d459037e-bde9-4df8-8cf9-0d6818a3102a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:18.303274 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1404/2795] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:26.915299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=83,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:34.447098 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da4d58-0669-42b8-8d21-7bd53824dfe9 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:38:34.448545 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-72da4d58-0669-42b8-8d21-7bd53824dfe9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:34.527852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da4d58-0669-42b8-8d21-7bd53824dfe9 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:38:34.533814 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c843a7-cfcc-4e4a-afda-b173be7ede22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:38:34.534441 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c843a7-cfcc-4e4a-afda-b173be7ede22 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:34.534716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c843a7-cfcc-4e4a-afda-b173be7ede22 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:34.534982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c843a7-cfcc-4e4a-afda-b173be7ede22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:34.536827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e7c843a7-cfcc-4e4a-afda-b173be7ede22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:34.550442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c843a7-cfcc-4e4a-afda-b173be7ede22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:38:34.552122 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c843a7-cfcc-4e4a-afda-b173be7ede22 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:38:34.562817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1392/2796] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:38:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:38:35.981993 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625b059-4cd3-4b30-8def-f1ee331c5263 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:38:35.982521 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625b059-4cd3-4b30-8def-f1ee331c5263 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:38:35.982635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625b059-4cd3-4b30-8def-f1ee331c5263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:38:36.002884 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b625b059-4cd3-4b30-8def-f1ee331c5263 None None] MFA Rules not processed for user `53fb0b1bfcb6449089f3162f88145a14`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:38:36.004081 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b625b059-4cd3-4b30-8def-f1ee331c5263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:38:36.011014 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1405/2797] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:38:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:38:44.558223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=34 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:38:45.998023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:39:00.349465 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1eb6d8-995c-4929-a3e0-001670f491ab None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:00.349936 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1eb6d8-995c-4929-a3e0-001670f491ab None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:00.350097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1eb6d8-995c-4929-a3e0-001670f491ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:00.369491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5f1eb6d8-995c-4929-a3e0-001670f491ab None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:39:00.370710 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5f1eb6d8-995c-4929-a3e0-001670f491ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:39:00.425673 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1393/2798] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:39:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:39:00.472411 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625b059-4cd3-4b30-8def-f1ee331c5263 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:39:00.473811 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b625b059-4cd3-4b30-8def-f1ee331c5263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:39:00.541027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625b059-4cd3-4b30-8def-f1ee331c5263 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:39:00.545848 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb7c86-01e7-4bae-bfae-4871e522d888 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:39:00.546478 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb7c86-01e7-4bae-bfae-4871e522d888 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:00.546755 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb7c86-01e7-4bae-bfae-4871e522d888 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:00.547023 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb7c86-01e7-4bae-bfae-4871e522d888 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:00.554855 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb7c86-01e7-4bae-bfae-4871e522d888 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:39:00.556461 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb7c86-01e7-4bae-bfae-4871e522d888 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:39:00.564781 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1406/2799] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:39:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:39:06.219269 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1eb6d8-995c-4929-a3e0-001670f491ab None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:39:06.221042 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5f1eb6d8-995c-4929-a3e0-001670f491ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:39:06.236470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1eb6d8-995c-4929-a3e0-001670f491ab None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:39:06.241132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8852d1-9303-45c3-afe2-0245b75f90fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:39:06.241877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8852d1-9303-45c3-afe2-0245b75f90fe service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:06.242442 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8852d1-9303-45c3-afe2-0245b75f90fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:06.242923 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8852d1-9303-45c3-afe2-0245b75f90fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:06.244752 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ce8852d1-9303-45c3-afe2-0245b75f90fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:39:06.260668 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8852d1-9303-45c3-afe2-0245b75f90fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:39:06.262328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8852d1-9303-45c3-afe2-0245b75f90fe service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:39:06.289631 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1394/2800] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:39:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:39:07.491196 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb7c86-01e7-4bae-bfae-4871e522d888 service swift] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:39:07.504320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb7c86-01e7-4bae-bfae-4871e522d888 service swift] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:39:07.508823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d6352-896b-47ad-8c46-44931e833cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:39:07.509700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d6352-896b-47ad-8c46-44931e833cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:07.510271 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d6352-896b-47ad-8c46-44931e833cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:07.510800 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d6352-896b-47ad-8c46-44931e833cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:07.513654 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-283d6352-896b-47ad-8c46-44931e833cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:39:07.530344 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d6352-896b-47ad-8c46-44931e833cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:39:07.531948 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283d6352-896b-47ad-8c46-44931e833cc6 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:39:07.562325 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1407/2801] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:39:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:39:16.134332 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab0b98-c4c3-4a28-8d89-530bff71fd5f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:16.134876 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab0b98-c4c3-4a28-8d89-530bff71fd5f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:16.134989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab0b98-c4c3-4a28-8d89-530bff71fd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:16.154997 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ffab0b98-c4c3-4a28-8d89-530bff71fd5f None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:39:16.157083 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ffab0b98-c4c3-4a28-8d89-530bff71fd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:39:16.164544 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1395/2802] 10.222.0.55 () {58 vars in 967 bytes} [Tue Sep 27 17:39:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:39:16.180604 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d6352-896b-47ad-8c46-44931e833cc6 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:39:16.191126 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283d6352-896b-47ad-8c46-44931e833cc6 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:39:16.196142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef3d4c-6ae7-49b5-a805-d5ac886597f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:39:16.197475 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef3d4c-6ae7-49b5-a805-d5ac886597f4 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:16.198004 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef3d4c-6ae7-49b5-a805-d5ac886597f4 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:16.198291 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef3d4c-6ae7-49b5-a805-d5ac886597f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:16.208391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef3d4c-6ae7-49b5-a805-d5ac886597f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:39:16.209959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef3d4c-6ae7-49b5-a805-d5ac886597f4 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:39:16.219423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1408/2803] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:39:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:39:18.420561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab0b98-c4c3-4a28-8d89-530bff71fd5f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:39:18.432411 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab0b98-c4c3-4a28-8d89-530bff71fd5f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:39:18.436520 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:39:18.437130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:18.437398 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:18.437678 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] PATH_INFO: `/v3/users/53fb0b1bfcb6449089f3162f88145a14` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:18.438808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53fb0b1bfcb6449089f3162f88145a14)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:39:18.441242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:39:18.486532 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53fb0b1bfcb6449089f3162f88145a14'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:18.487753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53fb0b1bfcb6449089f3162f88145a14'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:18.494628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53fb0b1bfcb6449089f3162f88145a14'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:18.501817 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53fb0b1bfcb6449089f3162f88145a14'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:18.514087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53fb0b1bfcb6449089f3162f88145a14'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:18.520595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1396/2804] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:39:18 2022] DELETE /identity/v3/users/53fb0b1bfcb6449089f3162f88145a14 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:39:18.686010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef3d4c-6ae7-49b5-a805-d5ac886597f4 service swift] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:39:18.697164 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef3d4c-6ae7-49b5-a805-d5ac886597f4 service swift] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:39:18.754827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:39:18.755470 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:18.755756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:18.756100 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] PATH_INFO: `/v3/projects/7a5cb427a38643a9b547e710d1c9caf5` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:18.757419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a5cb427a38643a9b547e710d1c9caf5)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:39:18.759251 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:39:18.810389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a5cb427a38643a9b547e710d1c9caf5', 'request_id': 'req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:18.816554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a5cb427a38643a9b547e710d1c9caf5', 'request_id': 'req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:18.823098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a5cb427a38643a9b547e710d1c9caf5', 'request_id': 'req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:18.825638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:39:18.839064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] The token cache is being invalidate because project 7a5cb427a38643a9b547e710d1c9caf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:39:18.839634 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:18.840280 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:18.841784 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1409/2805] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:39:18 2022] DELETE /identity/v3/projects/7a5cb427a38643a9b547e710d1c9caf5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:39:19.002637 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:39:19.005028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:39:19.072237 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14b5bb-f769-4454-a27d-441b24e8ede4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:39:19.081598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:39:19.128062 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784af4f-90e0-46ed-95c8-68c05435756e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:39:19.128767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784af4f-90e0-46ed-95c8-68c05435756e service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:19.129043 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784af4f-90e0-46ed-95c8-68c05435756e service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:19.129308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784af4f-90e0-46ed-95c8-68c05435756e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:19.131118 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8784af4f-90e0-46ed-95c8-68c05435756e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:39:19.191814 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784af4f-90e0-46ed-95c8-68c05435756e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:39:19.193230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784af4f-90e0-46ed-95c8-68c05435756e service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:39:19.202887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1397/2806] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:39:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:39:25.267931 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:39:25.269419 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:39:25.284978 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc20727-6533-4c0a-a0a3-9d832f40d1b2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:39:25.289463 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:39:25.290113 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:25.290391 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:25.290661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] PATH_INFO: `/v3/users/cac1528c35fb464db00b5d18396b8eac` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:25.291764 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cac1528c35fb464db00b5d18396b8eac)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:39:25.293655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:39:25.340893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cac1528c35fb464db00b5d18396b8eac'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.348130 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cac1528c35fb464db00b5d18396b8eac'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.348763 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cac1528c35fb464db00b5d18396b8eac'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.357356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cac1528c35fb464db00b5d18396b8eac'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.370311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cac1528c35fb464db00b5d18396b8eac'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.375970 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1410/2807] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:39:25 2022] DELETE /identity/v3/users/cac1528c35fb464db00b5d18396b8eac => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:39:25.553732 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784af4f-90e0-46ed-95c8-68c05435756e service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:39:25.563714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784af4f-90e0-46ed-95c8-68c05435756e service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:39:25.620361 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:39:25.620979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:25.621250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:25.621516 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] PATH_INFO: `/v3/projects/f4db1a85e2504cb9956a6461b5072aa3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:25.622646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4db1a85e2504cb9956a6461b5072aa3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:39:25.624523 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:39:25.669794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4db1a85e2504cb9956a6461b5072aa3', 'request_id': 'req-6669da49-c7ec-42d2-b6c0-b195943ae0bd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.675192 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4db1a85e2504cb9956a6461b5072aa3', 'request_id': 'req-6669da49-c7ec-42d2-b6c0-b195943ae0bd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.681190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4db1a85e2504cb9956a6461b5072aa3', 'request_id': 'req-6669da49-c7ec-42d2-b6c0-b195943ae0bd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.683650 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:39:25.694183 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] The token cache is being invalidate because project f4db1a85e2504cb9956a6461b5072aa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:39:25.694407 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.695047 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.696539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1398/2808] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:39:25 2022] DELETE /identity/v3/projects/f4db1a85e2504cb9956a6461b5072aa3 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:39:25.700659 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:39:25.702052 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:39:25.715226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc99db-c4e3-414b-b278-511e6aa483c2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:39:25.770520 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:39:25.771143 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:25.771420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:25.771692 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] PATH_INFO: `/v3/users/6f41a64b57de459c8bb764bf7eb3737c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:25.772814 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f41a64b57de459c8bb764bf7eb3737c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:39:25.774780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:39:25.818848 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f41a64b57de459c8bb764bf7eb3737c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.825528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f41a64b57de459c8bb764bf7eb3737c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.826213 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f41a64b57de459c8bb764bf7eb3737c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.836503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f41a64b57de459c8bb764bf7eb3737c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.841605 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:39:25.852443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f41a64b57de459c8bb764bf7eb3737c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:25.858514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1411/2809] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:39:25 2022] DELETE /identity/v3/users/6f41a64b57de459c8bb764bf7eb3737c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:39:26.026765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:39:26.028280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:39:26.042718 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669da49-c7ec-42d2-b6c0-b195943ae0bd admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:39:26.101387 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:39:26.102160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:26.102454 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:26.102707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] PATH_INFO: `/v3/projects/c26e4ed5e0bc4eb5ad8b3f79e56b2424` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:26.103895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26e4ed5e0bc4eb5ad8b3f79e56b2424)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:39:26.106773 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:39:26.153535 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26e4ed5e0bc4eb5ad8b3f79e56b2424', 'request_id': 'req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:26.158993 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26e4ed5e0bc4eb5ad8b3f79e56b2424', 'request_id': 'req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:26.165028 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26e4ed5e0bc4eb5ad8b3f79e56b2424', 'request_id': 'req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:26.176973 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] The token cache is being invalidate because project c26e4ed5e0bc4eb5ad8b3f79e56b2424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:39:26.177260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:26.177875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:39:26.179366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1399/2810] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:39:26 2022] DELETE /identity/v3/projects/c26e4ed5e0bc4eb5ad8b3f79e56b2424 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:39:35.854906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=2,keystone:SELECT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:39:36.174346 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=8,keystone:SELECT=53,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:39:56.733984 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:39:56.735571 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:39:56.794003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6d3c5-a393-49b2-a613-a75ee96a8acb admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:39:56.840147 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19018960-8207-465f-a6a2-40e91672c96e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:39:56.840749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19018960-8207-465f-a6a2-40e91672c96e service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:39:56.841027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19018960-8207-465f-a6a2-40e91672c96e service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:39:56.841296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19018960-8207-465f-a6a2-40e91672c96e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:39:56.843020 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-19018960-8207-465f-a6a2-40e91672c96e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:39:56.852700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19018960-8207-465f-a6a2-40e91672c96e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:39:56.854142 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19018960-8207-465f-a6a2-40e91672c96e service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:39:56.861965 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1412/2811] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:39:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:40:06.858383 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=46 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:41:05.803101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:41:05.804539 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:41:05.893393 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7ab2a-ba4c-4f71-8b99-eba5472dacf3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:41:05.899521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de780765-cadc-4164-b3c8-b26bff3848e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:41:05.900137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de780765-cadc-4164-b3c8-b26bff3848e9 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:41:05.900408 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de780765-cadc-4164-b3c8-b26bff3848e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:41:05.900674 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de780765-cadc-4164-b3c8-b26bff3848e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:41:05.902487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-de780765-cadc-4164-b3c8-b26bff3848e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:41:05.916218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de780765-cadc-4164-b3c8-b26bff3848e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:41:05.917880 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de780765-cadc-4164-b3c8-b26bff3848e9 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:41:05.930341 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1400/2812] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:41:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:41:09.380935 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d3042-2c38-43bc-afc1-44da94206143 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:41:09.381436 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d3042-2c38-43bc-afc1-44da94206143 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:41:09.381563 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d3042-2c38-43bc-afc1-44da94206143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:41:09.400776 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ae9d3042-2c38-43bc-afc1-44da94206143 None None] MFA Rules not processed for user `1b888fd409954909874b5443cfed2838`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:41:09.401964 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ae9d3042-2c38-43bc-afc1-44da94206143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:41:09.454667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1413/2813] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:41:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:41:09.753007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de780765-cadc-4164-b3c8-b26bff3848e9 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:41:09.754471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-de780765-cadc-4164-b3c8-b26bff3848e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:41:09.765428 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de780765-cadc-4164-b3c8-b26bff3848e9 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:41:09.769712 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430f972-51d4-4eda-a8f4-910fee767386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:41:09.770337 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430f972-51d4-4eda-a8f4-910fee767386 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:41:09.770608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430f972-51d4-4eda-a8f4-910fee767386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:41:09.770872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430f972-51d4-4eda-a8f4-910fee767386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:41:09.772502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9430f972-51d4-4eda-a8f4-910fee767386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:41:09.781616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430f972-51d4-4eda-a8f4-910fee767386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:41:09.783068 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430f972-51d4-4eda-a8f4-910fee767386 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:41:09.790822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1401/2814] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:41:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:41:10.121858 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d3042-2c38-43bc-afc1-44da94206143 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:41:10.124232 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ae9d3042-2c38-43bc-afc1-44da94206143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:41:10.195930 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d3042-2c38-43bc-afc1-44da94206143 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:41:10.200850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab8116-1126-4ddd-ab0c-6a95acc0579b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:41:10.201461 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab8116-1126-4ddd-ab0c-6a95acc0579b service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:41:10.201739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab8116-1126-4ddd-ab0c-6a95acc0579b service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:41:10.202010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab8116-1126-4ddd-ab0c-6a95acc0579b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:41:10.203762 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a2ab8116-1126-4ddd-ab0c-6a95acc0579b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:41:10.215119 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab8116-1126-4ddd-ab0c-6a95acc0579b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:41:10.216686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab8116-1126-4ddd-ab0c-6a95acc0579b service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:41:10.226479 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1414/2815] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:41:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:41:19.510922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430f972-51d4-4eda-a8f4-910fee767386 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:41:19.526195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430f972-51d4-4eda-a8f4-910fee767386 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:41:19.531868 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46020031-527e-48e5-b3ea-c4ac1036612c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:41:19.532623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46020031-527e-48e5-b3ea-c4ac1036612c service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:41:19.532968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46020031-527e-48e5-b3ea-c4ac1036612c service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:41:19.533309 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46020031-527e-48e5-b3ea-c4ac1036612c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:41:19.535180 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-46020031-527e-48e5-b3ea-c4ac1036612c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:41:19.547757 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46020031-527e-48e5-b3ea-c4ac1036612c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:41:19.549756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46020031-527e-48e5-b3ea-c4ac1036612c service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:41:19.560491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1402/2816] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:41:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:41:20.222282 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=57 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:41:29.556661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=46 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:42:23.762152 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab8116-1126-4ddd-ab0c-6a95acc0579b service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:42:23.763619 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a2ab8116-1126-4ddd-ab0c-6a95acc0579b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:42:23.856774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab8116-1126-4ddd-ab0c-6a95acc0579b service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:42:23.861876 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f54548-b37f-4ea0-9bd4-5919ec4c6963 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:42:23.862513 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f54548-b37f-4ea0-9bd4-5919ec4c6963 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:42:23.862790 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f54548-b37f-4ea0-9bd4-5919ec4c6963 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:42:23.863062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f54548-b37f-4ea0-9bd4-5919ec4c6963 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:42:23.864772 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-23f54548-b37f-4ea0-9bd4-5919ec4c6963 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:42:23.876243 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f54548-b37f-4ea0-9bd4-5919ec4c6963 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:42:23.877795 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f54548-b37f-4ea0-9bd4-5919ec4c6963 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:42:23.887562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1415/2817] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:42:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:42:33.883340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=37 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:42:38.248157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46020031-527e-48e5-b3ea-c4ac1036612c service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:42:38.249589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-46020031-527e-48e5-b3ea-c4ac1036612c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:42:38.327565 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46020031-527e-48e5-b3ea-c4ac1036612c service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:42:38.333753 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a70bfe-f8bf-4705-a67c-b31729034d00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5537185f42474cfd86fca70fa8beddd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:42:38.334390 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a70bfe-f8bf-4705-a67c-b31729034d00 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:42:38.334663 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a70bfe-f8bf-4705-a67c-b31729034d00 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:42:38.334929 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a70bfe-f8bf-4705-a67c-b31729034d00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:42:38.336616 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-99a70bfe-f8bf-4705-a67c-b31729034d00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:42:38.346707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a70bfe-f8bf-4705-a67c-b31729034d00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:42:38.348244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a70bfe-f8bf-4705-a67c-b31729034d00 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:42:38.356895 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1403/2818] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:42:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:42:48.352938 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=35 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:08.987321 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f54548-b37f-4ea0-9bd4-5919ec4c6963 service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:09.006145 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f54548-b37f-4ea0-9bd4-5919ec4c6963 service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:09.009711 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa241a-1a7d-4fb4-ad9e-9b0d5076b6e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:09.010334 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa241a-1a7d-4fb4-ad9e-9b0d5076b6e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:09.010612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa241a-1a7d-4fb4-ad9e-9b0d5076b6e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:09.010879 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa241a-1a7d-4fb4-ad9e-9b0d5076b6e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:09.012565 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-88aa241a-1a7d-4fb4-ad9e-9b0d5076b6e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:09.093149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa241a-1a7d-4fb4-ad9e-9b0d5076b6e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:09.094612 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa241a-1a7d-4fb4-ad9e-9b0d5076b6e9 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:09.113676 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1416/2819] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:11.900608 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a70bfe-f8bf-4705-a67c-b31729034d00 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:11.913998 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a70bfe-f8bf-4705-a67c-b31729034d00 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:11.919341 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac364db-b9fc-41b9-9f45-b1f3b34cb693 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:11.919944 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac364db-b9fc-41b9-9f45-b1f3b34cb693 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:11.920223 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac364db-b9fc-41b9-9f45-b1f3b34cb693 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:11.920525 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac364db-b9fc-41b9-9f45-b1f3b34cb693 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:11.922713 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-aac364db-b9fc-41b9-9f45-b1f3b34cb693 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:11.936744 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac364db-b9fc-41b9-9f45-b1f3b34cb693 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:11.938375 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac364db-b9fc-41b9-9f45-b1f3b34cb693 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:11.950327 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1404/2820] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:11.976434 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa241a-1a7d-4fb4-ad9e-9b0d5076b6e9 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:11.978816 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-88aa241a-1a7d-4fb4-ad9e-9b0d5076b6e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:11.996226 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa241a-1a7d-4fb4-ad9e-9b0d5076b6e9 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:12.000911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab25cb8-63fd-4aab-9f4c-61d76c423f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:12.001536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab25cb8-63fd-4aab-9f4c-61d76c423f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:12.001813 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab25cb8-63fd-4aab-9f4c-61d76c423f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:12.002232 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab25cb8-63fd-4aab-9f4c-61d76c423f50 admin admin] PATH_INFO: `/v3/users/1b888fd409954909874b5443cfed2838` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:12.003765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab25cb8-63fd-4aab-9f4c-61d76c423f50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b888fd409954909874b5443cfed2838)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:12.005783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab25cb8-63fd-4aab-9f4c-61d76c423f50 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:12.052244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8ab25cb8-63fd-4aab-9f4c-61d76c423f50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b888fd409954909874b5443cfed2838'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:12.059098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8ab25cb8-63fd-4aab-9f4c-61d76c423f50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b888fd409954909874b5443cfed2838'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:12.059780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8ab25cb8-63fd-4aab-9f4c-61d76c423f50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b888fd409954909874b5443cfed2838'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:12.067150 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8ab25cb8-63fd-4aab-9f4c-61d76c423f50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b888fd409954909874b5443cfed2838'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:12.079065 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8ab25cb8-63fd-4aab-9f4c-61d76c423f50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b888fd409954909874b5443cfed2838'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:12.084959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1417/2821] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:43:11 2022] DELETE /identity/v3/users/1b888fd409954909874b5443cfed2838 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:43:12.255531 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac364db-b9fc-41b9-9f45-b1f3b34cb693 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:12.269723 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac364db-b9fc-41b9-9f45-b1f3b34cb693 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:12.335108 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28155ae-7005-4568-b8ac-7c3d91c54b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:12.335762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28155ae-7005-4568-b8ac-7c3d91c54b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:12.336036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28155ae-7005-4568-b8ac-7c3d91c54b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:12.336419 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28155ae-7005-4568-b8ac-7c3d91c54b80 admin admin] PATH_INFO: `/v3/projects/0bd65ea59a6d41219ecf99f5a554056a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:12.337533 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28155ae-7005-4568-b8ac-7c3d91c54b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd65ea59a6d41219ecf99f5a554056a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:12.339497 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28155ae-7005-4568-b8ac-7c3d91c54b80 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:12.384001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c28155ae-7005-4568-b8ac-7c3d91c54b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd65ea59a6d41219ecf99f5a554056a', 'request_id': 'req-c28155ae-7005-4568-b8ac-7c3d91c54b80'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:12.389321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c28155ae-7005-4568-b8ac-7c3d91c54b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd65ea59a6d41219ecf99f5a554056a', 'request_id': 'req-c28155ae-7005-4568-b8ac-7c3d91c54b80'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:12.395832 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c28155ae-7005-4568-b8ac-7c3d91c54b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd65ea59a6d41219ecf99f5a554056a', 'request_id': 'req-c28155ae-7005-4568-b8ac-7c3d91c54b80'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:12.408001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c28155ae-7005-4568-b8ac-7c3d91c54b80 admin admin] The token cache is being invalidate because project 0bd65ea59a6d41219ecf99f5a554056a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:43:12.408293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c28155ae-7005-4568-b8ac-7c3d91c54b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:12.409574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c28155ae-7005-4568-b8ac-7c3d91c54b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:12.411090 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1405/2822] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:43:12 2022] DELETE /identity/v3/projects/0bd65ea59a6d41219ecf99f5a554056a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:12.416727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecff6eb-dabb-4f1f-9a06-dc29c260fe8d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:12.417010 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecff6eb-dabb-4f1f-9a06-dc29c260fe8d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:12.417285 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecff6eb-dabb-4f1f-9a06-dc29c260fe8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:12.456306 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:12.477549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5ecff6eb-dabb-4f1f-9a06-dc29c260fe8d None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:12.478864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5ecff6eb-dabb-4f1f-9a06-dc29c260fe8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:12.539003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1418/2823] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:43:12.546909 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb91677-de63-4c10-852a-1064621f082b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:12.547193 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb91677-de63-4c10-852a-1064621f082b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:12.547496 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb91677-de63-4c10-852a-1064621f082b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:12.567831 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ceb91677-de63-4c10-852a-1064621f082b None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:12.569027 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ceb91677-de63-4c10-852a-1064621f082b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:12.577132 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1406/2824] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:12.581655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecff6eb-dabb-4f1f-9a06-dc29c260fe8d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:12.591182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecff6eb-dabb-4f1f-9a06-dc29c260fe8d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:12.595722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd5827-a154-419d-8b4d-25afde536e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:12.596860 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd5827-a154-419d-8b4d-25afde536e00 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:12.597132 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd5827-a154-419d-8b4d-25afde536e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:12.597394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd5827-a154-419d-8b4d-25afde536e00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:12.598029 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd5827-a154-419d-8b4d-25afde536e00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:43:12.598354 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd5827-a154-419d-8b4d-25afde536e00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:12.599939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd5827-a154-419d-8b4d-25afde536e00 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:12.611200 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1419/2825] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:43:12.615716 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb91677-de63-4c10-852a-1064621f082b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:12.624978 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb91677-de63-4c10-852a-1064621f082b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:12.633521 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920fcbc-73fd-4b2f-93a4-01c9510f6674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:12.634230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920fcbc-73fd-4b2f-93a4-01c9510f6674 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:12.634729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920fcbc-73fd-4b2f-93a4-01c9510f6674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:12.635227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920fcbc-73fd-4b2f-93a4-01c9510f6674 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:12.635928 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920fcbc-73fd-4b2f-93a4-01c9510f6674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:12.637858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920fcbc-73fd-4b2f-93a4-01c9510f6674 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:12.665082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1407/2826] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:43:12 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:43:12.669278 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd5827-a154-419d-8b4d-25afde536e00 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:12.679241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd5827-a154-419d-8b4d-25afde536e00 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:12.734723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc463c2-7312-49c8-93da-157a8643ae33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:12.735241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc463c2-7312-49c8-93da-157a8643ae33 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:12.735535 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc463c2-7312-49c8-93da-157a8643ae33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:12.735799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc463c2-7312-49c8-93da-157a8643ae33 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:12.736424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc463c2-7312-49c8-93da-157a8643ae33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:12.738379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc463c2-7312-49c8-93da-157a8643ae33 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:12.759629 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1420/2827] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:43:12 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:12.763729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920fcbc-73fd-4b2f-93a4-01c9510f6674 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:12.774470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920fcbc-73fd-4b2f-93a4-01c9510f6674 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:12.778998 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f112e-19a1-4d8c-a1dc-84b926dc3503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:12.779578 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f112e-19a1-4d8c-a1dc-84b926dc3503 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:12.779852 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f112e-19a1-4d8c-a1dc-84b926dc3503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:12.780119 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f112e-19a1-4d8c-a1dc-84b926dc3503 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:12.780744 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f112e-19a1-4d8c-a1dc-84b926dc3503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:12.782389 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f112e-19a1-4d8c-a1dc-84b926dc3503 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:12.792538 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1408/2828] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:12.796324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc463c2-7312-49c8-93da-157a8643ae33 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:12.806662 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc463c2-7312-49c8-93da-157a8643ae33 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:12.811217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ca6cc-1500-4795-b754-0b8900a56c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:12.811807 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ca6cc-1500-4795-b754-0b8900a56c9b admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:12.812082 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ca6cc-1500-4795-b754-0b8900a56c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:12.812347 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ca6cc-1500-4795-b754-0b8900a56c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:12.812980 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ca6cc-1500-4795-b754-0b8900a56c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:12.815064 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ca6cc-1500-4795-b754-0b8900a56c9b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:12.825827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1421/2829] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:12.829890 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f112e-19a1-4d8c-a1dc-84b926dc3503 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:12.840216 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f112e-19a1-4d8c-a1dc-84b926dc3503 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:12.844735 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22914773-918f-4404-a9a8-ff1f4b343e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:12.845354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22914773-918f-4404-a9a8-ff1f4b343e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:12.845628 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22914773-918f-4404-a9a8-ff1f4b343e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:12.845918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22914773-918f-4404-a9a8-ff1f4b343e30 admin admin] PATH_INFO: `/v3/projects/50cbb0968a804488a40b0c46386df16a/users/ed44239dffac4067bbe165b5e1fe9b65/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:12.859803 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22914773-918f-4404-a9a8-ff1f4b343e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50cbb0968a804488a40b0c46386df16a, user_id=ed44239dffac4067bbe165b5e1fe9b65, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:12.862488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22914773-918f-4404-a9a8-ff1f4b343e30 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:12.875515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1409/2830] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:12 2022] PUT /identity/v3/projects/50cbb0968a804488a40b0c46386df16a/users/ed44239dffac4067bbe165b5e1fe9b65/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:12.886258 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c67e4-e590-47c6-b10f-e2ea44247ed2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:12.886545 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c67e4-e590-47c6-b10f-e2ea44247ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:12.886820 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c67e4-e590-47c6-b10f-e2ea44247ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:12.914541 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a83c67e4-e590-47c6-b10f-e2ea44247ed2 None None] MFA Rules not processed for user `ed44239dffac4067bbe165b5e1fe9b65`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:12.915757 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a83c67e4-e590-47c6-b10f-e2ea44247ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:12.967747 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:12.978533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1422/2831] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:43:12.982949 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22914773-918f-4404-a9a8-ff1f4b343e30 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:12.996858 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:13.045883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22914773-918f-4404-a9a8-ff1f4b343e30 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:13.051039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4ad5e-d2c9-40bd-a7b2-2f1038df2f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:13.051629 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4ad5e-d2c9-40bd-a7b2-2f1038df2f1d admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:13.051907 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4ad5e-d2c9-40bd-a7b2-2f1038df2f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:13.052177 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4ad5e-d2c9-40bd-a7b2-2f1038df2f1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:13.052790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4ad5e-d2c9-40bd-a7b2-2f1038df2f1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:13.054734 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4ad5e-d2c9-40bd-a7b2-2f1038df2f1d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:13.080849 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1410/2832] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:43:12 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:13.085006 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c67e4-e590-47c6-b10f-e2ea44247ed2 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:13.095835 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c67e4-e590-47c6-b10f-e2ea44247ed2 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:13.152827 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a01bdd-f0fb-462d-80df-df078357afd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:13.153423 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a01bdd-f0fb-462d-80df-df078357afd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:13.153702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a01bdd-f0fb-462d-80df-df078357afd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:13.153973 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a01bdd-f0fb-462d-80df-df078357afd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:13.154627 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a01bdd-f0fb-462d-80df-df078357afd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:13.156534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a01bdd-f0fb-462d-80df-df078357afd5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:13.177238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1423/2833] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:43:13 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:13.181380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4ad5e-d2c9-40bd-a7b2-2f1038df2f1d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:13.192079 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4ad5e-d2c9-40bd-a7b2-2f1038df2f1d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:13.196570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcf672-f955-4ecd-b010-99d522a0ca9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:13.197149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcf672-f955-4ecd-b010-99d522a0ca9c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:13.197422 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcf672-f955-4ecd-b010-99d522a0ca9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:13.197709 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcf672-f955-4ecd-b010-99d522a0ca9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:13.198396 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabcf672-f955-4ecd-b010-99d522a0ca9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:13.199959 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabcf672-f955-4ecd-b010-99d522a0ca9c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:13.210810 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1411/2834] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:13.214906 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a01bdd-f0fb-462d-80df-df078357afd5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:13.224771 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a01bdd-f0fb-462d-80df-df078357afd5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:13.229289 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ccf4a-2037-4146-84d2-14433ba33d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:13.229922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ccf4a-2037-4146-84d2-14433ba33d71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:13.230234 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ccf4a-2037-4146-84d2-14433ba33d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:13.230505 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ccf4a-2037-4146-84d2-14433ba33d71 admin admin] PATH_INFO: `/v3/projects/4516d87670ae488eb13c6a65eab4dc4a/users/c5d01a622b824458a1dd38ed988294e4/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:13.245504 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ccf4a-2037-4146-84d2-14433ba33d71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4516d87670ae488eb13c6a65eab4dc4a, user_id=c5d01a622b824458a1dd38ed988294e4, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:13.248220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ccf4a-2037-4146-84d2-14433ba33d71 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:13.261562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1424/2835] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:13 2022] PUT /identity/v3/projects/4516d87670ae488eb13c6a65eab4dc4a/users/c5d01a622b824458a1dd38ed988294e4/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:13.265466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcf672-f955-4ecd-b010-99d522a0ca9c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:13.275399 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcf672-f955-4ecd-b010-99d522a0ca9c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:13.332875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c9dc1-05c7-4521-9500-211b38bbbefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:13.333473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c9dc1-05c7-4521-9500-211b38bbbefd admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:13.333747 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c9dc1-05c7-4521-9500-211b38bbbefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:13.334016 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c9dc1-05c7-4521-9500-211b38bbbefd admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:13.334678 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c9dc1-05c7-4521-9500-211b38bbbefd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:13.336250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c9dc1-05c7-4521-9500-211b38bbbefd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:13.345741 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1412/2836] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:13.350116 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ccf4a-2037-4146-84d2-14433ba33d71 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:13.360309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ccf4a-2037-4146-84d2-14433ba33d71 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:13.365562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485763b-780e-4c9e-be55-67aa5a80b437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:13.366515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485763b-780e-4c9e-be55-67aa5a80b437 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:13.366791 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485763b-780e-4c9e-be55-67aa5a80b437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:13.367062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485763b-780e-4c9e-be55-67aa5a80b437 admin admin] PATH_INFO: `/v3/projects/4516d87670ae488eb13c6a65eab4dc4a/users/c5d01a622b824458a1dd38ed988294e4/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:13.369057 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485763b-780e-4c9e-be55-67aa5a80b437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4516d87670ae488eb13c6a65eab4dc4a, user_id=c5d01a622b824458a1dd38ed988294e4, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:13.371734 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485763b-780e-4c9e-be55-67aa5a80b437 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:13.382235 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1425/2837] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:13 2022] PUT /identity/v3/projects/4516d87670ae488eb13c6a65eab4dc4a/users/c5d01a622b824458a1dd38ed988294e4/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:13.393842 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada3a1-532e-4a4d-8890-d9c2cc4208ac None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:13.394153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada3a1-532e-4a4d-8890-d9c2cc4208ac None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:13.394430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cada3a1-532e-4a4d-8890-d9c2cc4208ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:13.421801 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2cada3a1-532e-4a4d-8890-d9c2cc4208ac None None] MFA Rules not processed for user `c5d01a622b824458a1dd38ed988294e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:13.423039 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2cada3a1-532e-4a4d-8890-d9c2cc4208ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:13.489597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1413/2838] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:13.500774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485763b-780e-4c9e-be55-67aa5a80b437 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:13.502488 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9485763b-780e-4c9e-be55-67aa5a80b437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:13.578785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485763b-780e-4c9e-be55-67aa5a80b437 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:13.582429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72c7fe-d9c3-41a6-9e88-2c6e9e23f646 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:13.583021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72c7fe-d9c3-41a6-9e88-2c6e9e23f646 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:13.583294 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72c7fe-d9c3-41a6-9e88-2c6e9e23f646 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:13.583561 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72c7fe-d9c3-41a6-9e88-2c6e9e23f646 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:13.592208 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72c7fe-d9c3-41a6-9e88-2c6e9e23f646 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:13.593624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72c7fe-d9c3-41a6-9e88-2c6e9e23f646 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:13.603336 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1426/2839] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:14.944541 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cada3a1-532e-4a4d-8890-d9c2cc4208ac None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:14.953883 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:15.005400 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cada3a1-532e-4a4d-8890-d9c2cc4208ac None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:15.009847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:15.010490 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:15.010768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:15.011037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] PATH_INFO: `/v3/users/ed44239dffac4067bbe165b5e1fe9b65` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:15.012148 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed44239dffac4067bbe165b5e1fe9b65)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:15.014055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:15.058367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed44239dffac4067bbe165b5e1fe9b65'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.059001 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed44239dffac4067bbe165b5e1fe9b65'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.066409 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed44239dffac4067bbe165b5e1fe9b65'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.073491 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed44239dffac4067bbe165b5e1fe9b65'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.085768 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed44239dffac4067bbe165b5e1fe9b65'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.091488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1414/2840] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:43:14 2022] DELETE /identity/v3/users/ed44239dffac4067bbe165b5e1fe9b65 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:15.095889 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a81e7c-7f78-4ea5-84bd-10def6e2c5cf None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:15.096380 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a81e7c-7f78-4ea5-84bd-10def6e2c5cf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:15.096499 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a81e7c-7f78-4ea5-84bd-10def6e2c5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:15.115134 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f8a81e7c-7f78-4ea5-84bd-10def6e2c5cf None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:15.116574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f8a81e7c-7f78-4ea5-84bd-10def6e2c5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:15.141103 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:15.176945 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1427/2841] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:15.187011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:15.188455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:15.259087 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe2731-2088-4abe-a360-a273bd3e05ea admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:15.262736 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95ee0a-d42d-4497-8bfb-16e84fa1a9ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:15.263330 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ee0a-d42d-4497-8bfb-16e84fa1a9ac service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:15.263607 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ee0a-d42d-4497-8bfb-16e84fa1a9ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:15.263877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ee0a-d42d-4497-8bfb-16e84fa1a9ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:15.272829 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95ee0a-d42d-4497-8bfb-16e84fa1a9ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:15.274290 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95ee0a-d42d-4497-8bfb-16e84fa1a9ac service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:15.283855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1415/2842] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:15.553305 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a81e7c-7f78-4ea5-84bd-10def6e2c5cf None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:15.554888 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f8a81e7c-7f78-4ea5-84bd-10def6e2c5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:15.567828 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a81e7c-7f78-4ea5-84bd-10def6e2c5cf None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:15.572492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:15.573058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:15.573332 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:15.573598 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] PATH_INFO: `/v3/projects/50cbb0968a804488a40b0c46386df16a` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:15.574737 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50cbb0968a804488a40b0c46386df16a)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:15.576643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:15.624351 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50cbb0968a804488a40b0c46386df16a', 'request_id': 'req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.630620 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50cbb0968a804488a40b0c46386df16a', 'request_id': 'req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.636766 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50cbb0968a804488a40b0c46386df16a', 'request_id': 'req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.648670 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] The token cache is being invalidate because project 50cbb0968a804488a40b0c46386df16a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:43:15.648950 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.649588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.651179 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1428/2843] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:43:15 2022] DELETE /identity/v3/projects/50cbb0968a804488a40b0c46386df16a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:15.655192 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95ee0a-d42d-4497-8bfb-16e84fa1a9ac service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:15.665250 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95ee0a-d42d-4497-8bfb-16e84fa1a9ac service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:15.681737 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:15.722813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c73ef-aacd-4d40-a7a5-366dfa7b680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:15.723437 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c73ef-aacd-4d40-a7a5-366dfa7b680d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:15.723714 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c73ef-aacd-4d40-a7a5-366dfa7b680d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:15.723988 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c73ef-aacd-4d40-a7a5-366dfa7b680d admin admin] PATH_INFO: `/v3/users/c5d01a622b824458a1dd38ed988294e4` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:15.725112 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c73ef-aacd-4d40-a7a5-366dfa7b680d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5d01a622b824458a1dd38ed988294e4)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:15.727068 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c73ef-aacd-4d40-a7a5-366dfa7b680d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:15.773770 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-096c73ef-aacd-4d40-a7a5-366dfa7b680d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5d01a622b824458a1dd38ed988294e4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.774482 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-096c73ef-aacd-4d40-a7a5-366dfa7b680d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5d01a622b824458a1dd38ed988294e4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.781429 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-096c73ef-aacd-4d40-a7a5-366dfa7b680d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5d01a622b824458a1dd38ed988294e4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.788873 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-096c73ef-aacd-4d40-a7a5-366dfa7b680d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5d01a622b824458a1dd38ed988294e4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.801215 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-096c73ef-aacd-4d40-a7a5-366dfa7b680d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5d01a622b824458a1dd38ed988294e4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:15.807230 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1416/2844] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:43:15 2022] DELETE /identity/v3/users/c5d01a622b824458a1dd38ed988294e4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:16.078538 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:16.079975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:16.094254 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e0cf8-358e-42bd-86e2-1f998bf4c854 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:16.152024 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae23f2a-f8fe-4883-90e9-98c0ca04821a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:16.152682 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae23f2a-f8fe-4883-90e9-98c0ca04821a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:16.152920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae23f2a-f8fe-4883-90e9-98c0ca04821a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:16.153192 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae23f2a-f8fe-4883-90e9-98c0ca04821a admin admin] PATH_INFO: `/v3/projects/4516d87670ae488eb13c6a65eab4dc4a` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:16.154331 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae23f2a-f8fe-4883-90e9-98c0ca04821a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4516d87670ae488eb13c6a65eab4dc4a)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:16.156232 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae23f2a-f8fe-4883-90e9-98c0ca04821a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:16.200739 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-dae23f2a-f8fe-4883-90e9-98c0ca04821a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4516d87670ae488eb13c6a65eab4dc4a', 'request_id': 'req-dae23f2a-f8fe-4883-90e9-98c0ca04821a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:16.204415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:16.207300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-dae23f2a-f8fe-4883-90e9-98c0ca04821a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4516d87670ae488eb13c6a65eab4dc4a', 'request_id': 'req-dae23f2a-f8fe-4883-90e9-98c0ca04821a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:16.215265 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-dae23f2a-f8fe-4883-90e9-98c0ca04821a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4516d87670ae488eb13c6a65eab4dc4a', 'request_id': 'req-dae23f2a-f8fe-4883-90e9-98c0ca04821a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:16.228139 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-dae23f2a-f8fe-4883-90e9-98c0ca04821a admin admin] The token cache is being invalidate because project 4516d87670ae488eb13c6a65eab4dc4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:43:16.228427 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-dae23f2a-f8fe-4883-90e9-98c0ca04821a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:16.229021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-dae23f2a-f8fe-4883-90e9-98c0ca04821a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:16.230547 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1429/2845] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:43:16 2022] DELETE /identity/v3/projects/4516d87670ae488eb13c6a65eab4dc4a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:16.235354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093094bb-99e7-4d21-a79c-de32792710ed None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:16.235837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093094bb-99e7-4d21-a79c-de32792710ed None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:16.235958 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093094bb-99e7-4d21-a79c-de32792710ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:16.255154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-093094bb-99e7-4d21-a79c-de32792710ed None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:16.256342 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-093094bb-99e7-4d21-a79c-de32792710ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:16.312903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1417/2846] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:16.320872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050d263-3aa3-480f-bc8c-92d5645b6f7e None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:16.321154 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050d263-3aa3-480f-bc8c-92d5645b6f7e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:16.321424 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050d263-3aa3-480f-bc8c-92d5645b6f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:16.341329 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5050d263-3aa3-480f-bc8c-92d5645b6f7e None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:16.342493 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5050d263-3aa3-480f-bc8c-92d5645b6f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:16.350839 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1430/2847] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:16.354964 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093094bb-99e7-4d21-a79c-de32792710ed None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:16.364451 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093094bb-99e7-4d21-a79c-de32792710ed None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:16.369401 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ad0fa-33c5-4153-bae0-4ef064113b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:16.369989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ad0fa-33c5-4153-bae0-4ef064113b9c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:16.370299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ad0fa-33c5-4153-bae0-4ef064113b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:16.370589 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ad0fa-33c5-4153-bae0-4ef064113b9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:16.371227 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ad0fa-33c5-4153-bae0-4ef064113b9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:43:16.371527 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ad0fa-33c5-4153-bae0-4ef064113b9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:16.373092 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ad0fa-33c5-4153-bae0-4ef064113b9c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:16.384908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1418/2848] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:43:16.388956 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050d263-3aa3-480f-bc8c-92d5645b6f7e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:16.398686 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050d263-3aa3-480f-bc8c-92d5645b6f7e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:16.403307 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093334af-11f8-4d65-855a-5bff283b6d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:16.403875 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093334af-11f8-4d65-855a-5bff283b6d67 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:16.404159 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093334af-11f8-4d65-855a-5bff283b6d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:16.404429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093334af-11f8-4d65-855a-5bff283b6d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:16.405058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093334af-11f8-4d65-855a-5bff283b6d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:16.407015 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093334af-11f8-4d65-855a-5bff283b6d67 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:16.432977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1431/2849] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:43:16 2022] POST /identity/v3/projects => generated 387 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:16.436963 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ad0fa-33c5-4153-bae0-4ef064113b9c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:16.447176 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ad0fa-33c5-4153-bae0-4ef064113b9c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:16.471172 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:16.504009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17076cbe-f555-4ff0-a044-62e6b824e48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:16.504652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17076cbe-f555-4ff0-a044-62e6b824e48c admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:16.504925 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17076cbe-f555-4ff0-a044-62e6b824e48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:16.505195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17076cbe-f555-4ff0-a044-62e6b824e48c admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:16.505822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17076cbe-f555-4ff0-a044-62e6b824e48c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:16.507812 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17076cbe-f555-4ff0-a044-62e6b824e48c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:16.527665 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1419/2850] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:43:16 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:16.531937 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093334af-11f8-4d65-855a-5bff283b6d67 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:16.542244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093334af-11f8-4d65-855a-5bff283b6d67 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:16.546869 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ded233-32b5-49ad-ba54-3ab7a62f5415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:16.547484 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ded233-32b5-49ad-ba54-3ab7a62f5415 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:16.547754 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ded233-32b5-49ad-ba54-3ab7a62f5415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:16.548019 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ded233-32b5-49ad-ba54-3ab7a62f5415 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:16.548657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ded233-32b5-49ad-ba54-3ab7a62f5415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:16.550298 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ded233-32b5-49ad-ba54-3ab7a62f5415 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:16.560035 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1432/2851] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:16.563763 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17076cbe-f555-4ff0-a044-62e6b824e48c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:16.573822 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17076cbe-f555-4ff0-a044-62e6b824e48c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:16.578468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9c572-c743-4b36-9386-4fe6e9fda2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:16.579041 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9c572-c743-4b36-9386-4fe6e9fda2cf admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:16.579316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9c572-c743-4b36-9386-4fe6e9fda2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:16.579583 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9c572-c743-4b36-9386-4fe6e9fda2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:16.580843 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9c572-c743-4b36-9386-4fe6e9fda2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:16.582426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9c572-c743-4b36-9386-4fe6e9fda2cf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:16.593281 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1420/2852] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:43:16.597207 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ded233-32b5-49ad-ba54-3ab7a62f5415 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:16.607097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ded233-32b5-49ad-ba54-3ab7a62f5415 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:16.611769 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668be25-3365-4dbc-ba88-acc84bed9903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:16.612409 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668be25-3365-4dbc-ba88-acc84bed9903 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:16.612687 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668be25-3365-4dbc-ba88-acc84bed9903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:16.612959 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668be25-3365-4dbc-ba88-acc84bed9903 admin admin] PATH_INFO: `/v3/projects/62813c2d5d664ee0ad889ef57bd21455/users/7f4a178ae20b40bdbbea44eb31101197/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:16.630290 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668be25-3365-4dbc-ba88-acc84bed9903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62813c2d5d664ee0ad889ef57bd21455, user_id=7f4a178ae20b40bdbbea44eb31101197, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:16.635898 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668be25-3365-4dbc-ba88-acc84bed9903 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:16.649718 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1433/2853] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:16 2022] PUT /identity/v3/projects/62813c2d5d664ee0ad889ef57bd21455/users/7f4a178ae20b40bdbbea44eb31101197/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:16.661203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1ce73-44cc-451c-a766-0aed904d366e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:16.661484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1ce73-44cc-451c-a766-0aed904d366e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:16.661751 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1ce73-44cc-451c-a766-0aed904d366e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:16.688434 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9fc1ce73-44cc-451c-a766-0aed904d366e None None] MFA Rules not processed for user `7f4a178ae20b40bdbbea44eb31101197`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:16.690157 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9fc1ce73-44cc-451c-a766-0aed904d366e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:16.753951 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1421/2854] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:16.758502 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668be25-3365-4dbc-ba88-acc84bed9903 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:16.818875 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668be25-3365-4dbc-ba88-acc84bed9903 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:16.823404 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc180d-a7b0-4676-92b2-6a9fcc84a2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:16.823994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc180d-a7b0-4676-92b2-6a9fcc84a2df admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:16.824266 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc180d-a7b0-4676-92b2-6a9fcc84a2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:16.824534 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc180d-a7b0-4676-92b2-6a9fcc84a2df admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:16.825156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc180d-a7b0-4676-92b2-6a9fcc84a2df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:16.827071 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc180d-a7b0-4676-92b2-6a9fcc84a2df admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:16.852501 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1434/2855] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:43:16 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:16.856505 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1ce73-44cc-451c-a766-0aed904d366e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:16.867949 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1ce73-44cc-451c-a766-0aed904d366e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:16.922288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8530a50-95e6-48bd-82b1-389510b88883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:16.922880 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8530a50-95e6-48bd-82b1-389510b88883 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:16.923220 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8530a50-95e6-48bd-82b1-389510b88883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:16.923486 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8530a50-95e6-48bd-82b1-389510b88883 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:16.924653 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8530a50-95e6-48bd-82b1-389510b88883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:16.926584 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8530a50-95e6-48bd-82b1-389510b88883 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:16.948234 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1422/2856] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:43:16 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:43:16.952245 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc180d-a7b0-4676-92b2-6a9fcc84a2df admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:16.962503 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc180d-a7b0-4676-92b2-6a9fcc84a2df admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:16.966999 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb9b7a-9ba8-4d4f-9eeb-9ea702a2fd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:16.967574 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb9b7a-9ba8-4d4f-9eeb-9ea702a2fd6a admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:16.967856 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb9b7a-9ba8-4d4f-9eeb-9ea702a2fd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:16.968118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cb9b7a-9ba8-4d4f-9eeb-9ea702a2fd6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:16.968748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb9b7a-9ba8-4d4f-9eeb-9ea702a2fd6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:16.970377 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cb9b7a-9ba8-4d4f-9eeb-9ea702a2fd6a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:16.980589 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1435/2857] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:16.984471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8530a50-95e6-48bd-82b1-389510b88883 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:16.995030 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8530a50-95e6-48bd-82b1-389510b88883 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:16.999744 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5204753-609b-4b8f-a92f-fa42fc98d45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:17.000380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5204753-609b-4b8f-a92f-fa42fc98d45c admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:17.000652 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5204753-609b-4b8f-a92f-fa42fc98d45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:17.000918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5204753-609b-4b8f-a92f-fa42fc98d45c admin admin] PATH_INFO: `/v3/projects/4ccca0fa293f4e1da80125850b6b231d/users/eb07d2baf8cf42238f337743aed85b2b/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:17.015827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5204753-609b-4b8f-a92f-fa42fc98d45c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ccca0fa293f4e1da80125850b6b231d, user_id=eb07d2baf8cf42238f337743aed85b2b, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:17.018476 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5204753-609b-4b8f-a92f-fa42fc98d45c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:17.026581 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:17.031983 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1423/2858] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:16 2022] PUT /identity/v3/projects/4ccca0fa293f4e1da80125850b6b231d/users/eb07d2baf8cf42238f337743aed85b2b/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:17.035824 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb9b7a-9ba8-4d4f-9eeb-9ea702a2fd6a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:17.045588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cb9b7a-9ba8-4d4f-9eeb-9ea702a2fd6a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:17.091150 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:17.100928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a167b-056c-4604-84af-1fb99f149c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:17.101528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a167b-056c-4604-84af-1fb99f149c84 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:17.101808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a167b-056c-4604-84af-1fb99f149c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:17.102101 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a167b-056c-4604-84af-1fb99f149c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:17.102804 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a167b-056c-4604-84af-1fb99f149c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:17.104381 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a167b-056c-4604-84af-1fb99f149c84 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:17.114566 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1436/2859] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:17.119266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5204753-609b-4b8f-a92f-fa42fc98d45c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:17.130576 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5204753-609b-4b8f-a92f-fa42fc98d45c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:17.135114 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53bdbc-35dd-4835-8542-3c43b9d7e787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:17.135740 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53bdbc-35dd-4835-8542-3c43b9d7e787 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:17.136009 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53bdbc-35dd-4835-8542-3c43b9d7e787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:17.136272 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53bdbc-35dd-4835-8542-3c43b9d7e787 admin admin] PATH_INFO: `/v3/projects/4ccca0fa293f4e1da80125850b6b231d/users/eb07d2baf8cf42238f337743aed85b2b/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:17.138168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53bdbc-35dd-4835-8542-3c43b9d7e787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ccca0fa293f4e1da80125850b6b231d, user_id=eb07d2baf8cf42238f337743aed85b2b, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:17.140868 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53bdbc-35dd-4835-8542-3c43b9d7e787 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:17.150724 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1424/2860] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:17 2022] PUT /identity/v3/projects/4ccca0fa293f4e1da80125850b6b231d/users/eb07d2baf8cf42238f337743aed85b2b/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:43:17.161738 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08849c1f-246e-4b54-a74d-73372b34c37a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:17.162268 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08849c1f-246e-4b54-a74d-73372b34c37a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:17.162376 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08849c1f-246e-4b54-a74d-73372b34c37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:17.189336 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-08849c1f-246e-4b54-a74d-73372b34c37a None None] MFA Rules not processed for user `eb07d2baf8cf42238f337743aed85b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:17.190593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-08849c1f-246e-4b54-a74d-73372b34c37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:17.264613 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1437/2861] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:17.275785 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53bdbc-35dd-4835-8542-3c43b9d7e787 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:17.277455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fb53bdbc-35dd-4835-8542-3c43b9d7e787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:17.334313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53bdbc-35dd-4835-8542-3c43b9d7e787 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:17.337918 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927137f4-0484-4994-b530-cd88e33c296d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:17.338533 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927137f4-0484-4994-b530-cd88e33c296d service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:17.338808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927137f4-0484-4994-b530-cd88e33c296d service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:17.339077 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927137f4-0484-4994-b530-cd88e33c296d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:17.348305 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927137f4-0484-4994-b530-cd88e33c296d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:17.349709 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927137f4-0484-4994-b530-cd88e33c296d service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:17.359073 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1425/2862] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:17.387821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08849c1f-246e-4b54-a74d-73372b34c37a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:17.447533 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08849c1f-246e-4b54-a74d-73372b34c37a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:17.452013 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:17.452621 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:17.452893 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:17.453157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] PATH_INFO: `/v3/users/7f4a178ae20b40bdbbea44eb31101197` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:17.454281 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4a178ae20b40bdbbea44eb31101197)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:17.456209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:17.501544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4a178ae20b40bdbbea44eb31101197'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:17.508710 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4a178ae20b40bdbbea44eb31101197'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:17.509403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4a178ae20b40bdbbea44eb31101197'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:17.516336 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4a178ae20b40bdbbea44eb31101197'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:17.526866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:17.529554 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4a178ae20b40bdbbea44eb31101197'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:17.537599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1438/2863] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:43:17 2022] DELETE /identity/v3/users/7f4a178ae20b40bdbbea44eb31101197 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:17.542045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35acfb-dfa7-4341-a9ae-e337feae1d61 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:17.542341 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35acfb-dfa7-4341-a9ae-e337feae1d61 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:17.542606 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35acfb-dfa7-4341-a9ae-e337feae1d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:17.560908 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4a35acfb-dfa7-4341-a9ae-e337feae1d61 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:17.562117 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4a35acfb-dfa7-4341-a9ae-e337feae1d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:17.620680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1426/2864] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:17.630412 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:17.631803 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:17.688429 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c7c8b-1c3d-4032-a5fe-6cdbe698d4c0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:17.692061 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368d9cb-b0f0-4225-96a1-7800faa5c05a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:17.692655 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368d9cb-b0f0-4225-96a1-7800faa5c05a service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:17.692933 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368d9cb-b0f0-4225-96a1-7800faa5c05a service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:17.693202 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368d9cb-b0f0-4225-96a1-7800faa5c05a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:17.701994 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368d9cb-b0f0-4225-96a1-7800faa5c05a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:17.703455 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368d9cb-b0f0-4225-96a1-7800faa5c05a service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:17.713238 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1439/2865] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:17.971218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35acfb-dfa7-4341-a9ae-e337feae1d61 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:17.972648 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4a35acfb-dfa7-4341-a9ae-e337feae1d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:17.984330 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35acfb-dfa7-4341-a9ae-e337feae1d61 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:17.988986 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:17.989808 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:17.990113 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:17.990522 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] PATH_INFO: `/v3/projects/62813c2d5d664ee0ad889ef57bd21455` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:17.991957 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62813c2d5d664ee0ad889ef57bd21455)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:17.993898 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:18.041344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62813c2d5d664ee0ad889ef57bd21455', 'request_id': 'req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.046850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62813c2d5d664ee0ad889ef57bd21455', 'request_id': 'req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.053114 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62813c2d5d664ee0ad889ef57bd21455', 'request_id': 'req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.065376 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] The token cache is being invalidate because project 62813c2d5d664ee0ad889ef57bd21455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:43:18.065662 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.066308 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.067755 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1427/2866] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:43:17 2022] DELETE /identity/v3/projects/62813c2d5d664ee0ad889ef57bd21455 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:18.071698 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368d9cb-b0f0-4225-96a1-7800faa5c05a service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:18.081340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368d9cb-b0f0-4225-96a1-7800faa5c05a service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:18.137529 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4699c5-11c4-4680-87d2-9a1d6aa654e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:18.138184 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4699c5-11c4-4680-87d2-9a1d6aa654e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:18.138487 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4699c5-11c4-4680-87d2-9a1d6aa654e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:18.138756 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4699c5-11c4-4680-87d2-9a1d6aa654e5 admin admin] PATH_INFO: `/v3/users/eb07d2baf8cf42238f337743aed85b2b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:18.139862 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4699c5-11c4-4680-87d2-9a1d6aa654e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb07d2baf8cf42238f337743aed85b2b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:18.141797 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4699c5-11c4-4680-87d2-9a1d6aa654e5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:18.186233 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ab4699c5-11c4-4680-87d2-9a1d6aa654e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb07d2baf8cf42238f337743aed85b2b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.193315 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ab4699c5-11c4-4680-87d2-9a1d6aa654e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb07d2baf8cf42238f337743aed85b2b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.193955 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ab4699c5-11c4-4680-87d2-9a1d6aa654e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb07d2baf8cf42238f337743aed85b2b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.201024 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ab4699c5-11c4-4680-87d2-9a1d6aa654e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb07d2baf8cf42238f337743aed85b2b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.213575 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ab4699c5-11c4-4680-87d2-9a1d6aa654e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb07d2baf8cf42238f337743aed85b2b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.219441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1440/2867] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:43:18 2022] DELETE /identity/v3/users/eb07d2baf8cf42238f337743aed85b2b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:18.489577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:18.491066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:18.506474 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1821ee2-caa2-4a35-bc7e-40dd4368dba0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:18.516656 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:18.566104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:18.566732 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:18.567011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:18.567284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06 admin admin] PATH_INFO: `/v3/projects/4ccca0fa293f4e1da80125850b6b231d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:18.568425 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ccca0fa293f4e1da80125850b6b231d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:18.570421 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:18.615602 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ccca0fa293f4e1da80125850b6b231d', 'request_id': 'req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.621336 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ccca0fa293f4e1da80125850b6b231d', 'request_id': 'req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.627948 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ccca0fa293f4e1da80125850b6b231d', 'request_id': 'req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.639794 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06 admin admin] The token cache is being invalidate because project 4ccca0fa293f4e1da80125850b6b231d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:43:18.640097 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.640726 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d20701f9-dd0c-40bf-8deb-19f83fc7fd06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:18.642300 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1428/2868] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:43:18 2022] DELETE /identity/v3/projects/4ccca0fa293f4e1da80125850b6b231d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:18.650007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67a29b-cfb0-4546-bde1-0400a276a1ec None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:18.650326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67a29b-cfb0-4546-bde1-0400a276a1ec None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:18.650603 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67a29b-cfb0-4546-bde1-0400a276a1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:18.670379 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ce67a29b-cfb0-4546-bde1-0400a276a1ec None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:18.671578 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ce67a29b-cfb0-4546-bde1-0400a276a1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:18.681328 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:18.730657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1441/2869] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:18.738920 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72d49a-7adc-4051-a75e-8ea51912ac38 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:18.739168 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72d49a-7adc-4051-a75e-8ea51912ac38 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:18.739433 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72d49a-7adc-4051-a75e-8ea51912ac38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:18.758998 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-db72d49a-7adc-4051-a75e-8ea51912ac38 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:18.760202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-db72d49a-7adc-4051-a75e-8ea51912ac38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:18.768530 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1429/2870] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:18.772674 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67a29b-cfb0-4546-bde1-0400a276a1ec None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:18.782332 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67a29b-cfb0-4546-bde1-0400a276a1ec None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:18.787571 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7094e-0d74-4df1-a586-f2ec80a6572d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:18.788166 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7094e-0d74-4df1-a586-f2ec80a6572d admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:18.788443 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7094e-0d74-4df1-a586-f2ec80a6572d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:18.788715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7094e-0d74-4df1-a586-f2ec80a6572d admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:18.789363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7094e-0d74-4df1-a586-f2ec80a6572d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:43:18.789667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7094e-0d74-4df1-a586-f2ec80a6572d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:18.791295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7094e-0d74-4df1-a586-f2ec80a6572d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:18.803327 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1442/2871] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:43:18.807636 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72d49a-7adc-4051-a75e-8ea51912ac38 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:18.816765 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db72d49a-7adc-4051-a75e-8ea51912ac38 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:18.821395 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3a527-a071-47ed-8289-f0e299c43a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:18.821974 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3a527-a071-47ed-8289-f0e299c43a42 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:18.822280 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3a527-a071-47ed-8289-f0e299c43a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:18.822549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3a527-a071-47ed-8289-f0e299c43a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:18.823166 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3a527-a071-47ed-8289-f0e299c43a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:18.825035 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3a527-a071-47ed-8289-f0e299c43a42 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:18.851428 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1430/2872] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:43:18 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:18.855507 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7094e-0d74-4df1-a586-f2ec80a6572d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:18.865853 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7094e-0d74-4df1-a586-f2ec80a6572d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:18.921308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aba88e-5374-41e6-bf9e-836be36e9618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:18.921956 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aba88e-5374-41e6-bf9e-836be36e9618 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:18.922247 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aba88e-5374-41e6-bf9e-836be36e9618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:18.922519 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aba88e-5374-41e6-bf9e-836be36e9618 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:18.923151 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aba88e-5374-41e6-bf9e-836be36e9618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:18.925653 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aba88e-5374-41e6-bf9e-836be36e9618 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:18.945385 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1443/2873] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:43:18 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 27 17:43:18.949512 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3a527-a071-47ed-8289-f0e299c43a42 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:18.959633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3a527-a071-47ed-8289-f0e299c43a42 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:18.964130 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96219716-e1e6-4305-86f0-207e4799852f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:18.964706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96219716-e1e6-4305-86f0-207e4799852f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:18.964979 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96219716-e1e6-4305-86f0-207e4799852f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:18.965244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96219716-e1e6-4305-86f0-207e4799852f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:18.965868 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96219716-e1e6-4305-86f0-207e4799852f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:18.967502 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96219716-e1e6-4305-86f0-207e4799852f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:18.977612 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1431/2874] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:18.981264 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aba88e-5374-41e6-bf9e-836be36e9618 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:18.991909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aba88e-5374-41e6-bf9e-836be36e9618 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:18.996398 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd6ab9-6f2e-46d7-8ccb-5c7983e9d2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:18.996969 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd6ab9-6f2e-46d7-8ccb-5c7983e9d2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:18.997246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd6ab9-6f2e-46d7-8ccb-5c7983e9d2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:18.997514 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd6ab9-6f2e-46d7-8ccb-5c7983e9d2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:18.998176 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd6ab9-6f2e-46d7-8ccb-5c7983e9d2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:18.999749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd6ab9-6f2e-46d7-8ccb-5c7983e9d2a5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:19.010156 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1444/2875] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:19.014215 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96219716-e1e6-4305-86f0-207e4799852f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:19.024707 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96219716-e1e6-4305-86f0-207e4799852f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:19.029284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bda0f5-776a-45cc-95f3-9c36eaa05b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:19.029904 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bda0f5-776a-45cc-95f3-9c36eaa05b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:19.030219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bda0f5-776a-45cc-95f3-9c36eaa05b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:19.030484 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bda0f5-776a-45cc-95f3-9c36eaa05b3d admin admin] PATH_INFO: `/v3/projects/3499aa5ed94f492d8520354e5d5dcb43/users/fffb363773ed4c0cada22c7dce5638ab/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:19.045045 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bda0f5-776a-45cc-95f3-9c36eaa05b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3499aa5ed94f492d8520354e5d5dcb43, user_id=fffb363773ed4c0cada22c7dce5638ab, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:19.047689 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bda0f5-776a-45cc-95f3-9c36eaa05b3d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:19.060299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1432/2876] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:19 2022] PUT /identity/v3/projects/3499aa5ed94f492d8520354e5d5dcb43/users/fffb363773ed4c0cada22c7dce5638ab/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:19.125032 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6e556-7b18-4267-bffe-9404aa88ecfd None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:19.125309 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6e556-7b18-4267-bffe-9404aa88ecfd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:19.125581 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6e556-7b18-4267-bffe-9404aa88ecfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:19.153246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d5c6e556-7b18-4267-bffe-9404aa88ecfd None None] MFA Rules not processed for user `fffb363773ed4c0cada22c7dce5638ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:19.154482 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d5c6e556-7b18-4267-bffe-9404aa88ecfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:19.215326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1445/2877] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:19.219777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bda0f5-776a-45cc-95f3-9c36eaa05b3d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:19.261570 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:19.281801 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bda0f5-776a-45cc-95f3-9c36eaa05b3d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:19.287061 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfd2df-6ef6-47c4-bf39-eb56b0bd2296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:19.287646 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfd2df-6ef6-47c4-bf39-eb56b0bd2296 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:19.287922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfd2df-6ef6-47c4-bf39-eb56b0bd2296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:19.288190 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfd2df-6ef6-47c4-bf39-eb56b0bd2296 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:19.288807 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cfd2df-6ef6-47c4-bf39-eb56b0bd2296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:19.290734 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cfd2df-6ef6-47c4-bf39-eb56b0bd2296 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:19.317324 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1433/2878] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:43:19 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:19.321340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6e556-7b18-4267-bffe-9404aa88ecfd None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:19.332479 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6e556-7b18-4267-bffe-9404aa88ecfd None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:19.340109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:19.389248 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387863a-37ba-4faf-b981-82b490401795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:19.389841 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387863a-37ba-4faf-b981-82b490401795 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:19.390140 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387863a-37ba-4faf-b981-82b490401795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:19.390413 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387863a-37ba-4faf-b981-82b490401795 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:19.391038 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387863a-37ba-4faf-b981-82b490401795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:19.393549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387863a-37ba-4faf-b981-82b490401795 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:19.414583 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1446/2879] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:43:19 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:19.418643 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfd2df-6ef6-47c4-bf39-eb56b0bd2296 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:19.428784 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfd2df-6ef6-47c4-bf39-eb56b0bd2296 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:19.433316 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceda9b8-930b-487a-8827-ad9adef28792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:19.433899 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceda9b8-930b-487a-8827-ad9adef28792 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:19.434209 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceda9b8-930b-487a-8827-ad9adef28792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:19.434480 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceda9b8-930b-487a-8827-ad9adef28792 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:19.435114 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceda9b8-930b-487a-8827-ad9adef28792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:19.436681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceda9b8-930b-487a-8827-ad9adef28792 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:19.447567 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1434/2880] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:19.451441 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387863a-37ba-4faf-b981-82b490401795 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:19.461780 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387863a-37ba-4faf-b981-82b490401795 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:19.466435 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9505e-1511-4c06-a0cb-115aa1ac012c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:19.467068 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9505e-1511-4c06-a0cb-115aa1ac012c admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:19.467346 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9505e-1511-4c06-a0cb-115aa1ac012c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:19.467617 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9505e-1511-4c06-a0cb-115aa1ac012c admin admin] PATH_INFO: `/v3/projects/ffb4b4b08ab0422bafb079ff2524e3b0/users/db9721b21d39425989878dbae37afcd8/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:19.482195 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9505e-1511-4c06-a0cb-115aa1ac012c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb4b4b08ab0422bafb079ff2524e3b0, user_id=db9721b21d39425989878dbae37afcd8, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:19.484840 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9505e-1511-4c06-a0cb-115aa1ac012c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:19.497403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1447/2881] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:19 2022] PUT /identity/v3/projects/ffb4b4b08ab0422bafb079ff2524e3b0/users/db9721b21d39425989878dbae37afcd8/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:19.501257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceda9b8-930b-487a-8827-ad9adef28792 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:19.512818 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceda9b8-930b-487a-8827-ad9adef28792 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:19.568769 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da368f48-e768-404f-beb2-ccb2afdd52c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:19.569360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da368f48-e768-404f-beb2-ccb2afdd52c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:19.569762 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da368f48-e768-404f-beb2-ccb2afdd52c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:19.570038 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da368f48-e768-404f-beb2-ccb2afdd52c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:19.570700 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da368f48-e768-404f-beb2-ccb2afdd52c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:19.573010 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da368f48-e768-404f-beb2-ccb2afdd52c3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:19.587032 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1435/2882] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:19.591169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9505e-1511-4c06-a0cb-115aa1ac012c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:19.601209 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9505e-1511-4c06-a0cb-115aa1ac012c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:19.605722 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124fec3-0384-4937-aa0d-05bf529bc9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:19.606395 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124fec3-0384-4937-aa0d-05bf529bc9e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:19.606665 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124fec3-0384-4937-aa0d-05bf529bc9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:19.606927 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124fec3-0384-4937-aa0d-05bf529bc9e5 admin admin] PATH_INFO: `/v3/projects/ffb4b4b08ab0422bafb079ff2524e3b0/users/db9721b21d39425989878dbae37afcd8/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:19.608803 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124fec3-0384-4937-aa0d-05bf529bc9e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb4b4b08ab0422bafb079ff2524e3b0, user_id=db9721b21d39425989878dbae37afcd8, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:19.611455 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124fec3-0384-4937-aa0d-05bf529bc9e5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:19.621483 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1448/2883] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:19 2022] PUT /identity/v3/projects/ffb4b4b08ab0422bafb079ff2524e3b0/users/db9721b21d39425989878dbae37afcd8/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:19.631649 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561c329-fd13-48f7-89de-b24422d9b770 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:19.631931 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561c329-fd13-48f7-89de-b24422d9b770 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:19.632201 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561c329-fd13-48f7-89de-b24422d9b770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:19.659840 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e561c329-fd13-48f7-89de-b24422d9b770 None None] MFA Rules not processed for user `db9721b21d39425989878dbae37afcd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:19.661065 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e561c329-fd13-48f7-89de-b24422d9b770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:19.726549 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1436/2884] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:19.739403 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124fec3-0384-4937-aa0d-05bf529bc9e5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:19.741086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4124fec3-0384-4937-aa0d-05bf529bc9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:19.797198 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124fec3-0384-4937-aa0d-05bf529bc9e5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:19.801336 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc483b-acf9-40b4-b3c4-10bccff3b563 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:19.801927 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc483b-acf9-40b4-b3c4-10bccff3b563 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:19.802244 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc483b-acf9-40b4-b3c4-10bccff3b563 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:19.802515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc483b-acf9-40b4-b3c4-10bccff3b563 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:19.811544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc483b-acf9-40b4-b3c4-10bccff3b563 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:19.812966 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc483b-acf9-40b4-b3c4-10bccff3b563 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:19.822861 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1449/2885] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:20.128336 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561c329-fd13-48f7-89de-b24422d9b770 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:20.137360 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561c329-fd13-48f7-89de-b24422d9b770 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:20.142212 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7706b3-8c32-4b52-97c1-55da601fe9eb tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db9721b21d39425989878dbae37afcd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb4b4b08ab0422bafb079ff2524e3b0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:20.142935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7706b3-8c32-4b52-97c1-55da601fe9eb tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:20.143595 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7706b3-8c32-4b52-97c1-55da601fe9eb tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:20.143874 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7706b3-8c32-4b52-97c1-55da601fe9eb tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:20.144526 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7706b3-8c32-4b52-97c1-55da601fe9eb tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:43:20.144830 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7706b3-8c32-4b52-97c1-55da601fe9eb tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:20.146466 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7706b3-8c32-4b52-97c1-55da601fe9eb tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:20.157687 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1437/2886] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:43:20.162301 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc483b-acf9-40b4-b3c4-10bccff3b563 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:20.171374 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc483b-acf9-40b4-b3c4-10bccff3b563 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:20.176543 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62a819-2220-4c1f-a1b0-86e693d72102 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db9721b21d39425989878dbae37afcd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb4b4b08ab0422bafb079ff2524e3b0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:20.177127 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62a819-2220-4c1f-a1b0-86e693d72102 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:20.177406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62a819-2220-4c1f-a1b0-86e693d72102 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:20.177677 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62a819-2220-4c1f-a1b0-86e693d72102 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:20.178341 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62a819-2220-4c1f-a1b0-86e693d72102 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:20.180225 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62a819-2220-4c1f-a1b0-86e693d72102 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:20.206115 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1450/2887] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:43:20 2022] POST /identity/v3/projects => generated 354 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:20.210005 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7706b3-8c32-4b52-97c1-55da601fe9eb tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:20.219902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7706b3-8c32-4b52-97c1-55da601fe9eb tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:20.240284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:20.278078 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cb24c-43be-48d4-9949-9200ec1892ad tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db9721b21d39425989878dbae37afcd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb4b4b08ab0422bafb079ff2524e3b0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:20.278704 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cb24c-43be-48d4-9949-9200ec1892ad tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:20.278989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cb24c-43be-48d4-9949-9200ec1892ad tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:20.279262 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cb24c-43be-48d4-9949-9200ec1892ad tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:20.279897 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cb24c-43be-48d4-9949-9200ec1892ad tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:43:20.280198 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cb24c-43be-48d4-9949-9200ec1892ad tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:20.281766 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cb24c-43be-48d4-9949-9200ec1892ad tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:20.293307 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1438/2888] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:43:20.330572 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62a819-2220-4c1f-a1b0-86e693d72102 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:20.340761 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62a819-2220-4c1f-a1b0-86e693d72102 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:20.345201 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404c785-c844-4a79-bfc9-ace78892a9f8 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db9721b21d39425989878dbae37afcd8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb4b4b08ab0422bafb079ff2524e3b0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:20.346340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404c785-c844-4a79-bfc9-ace78892a9f8 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:20.346615 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404c785-c844-4a79-bfc9-ace78892a9f8 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:20.346881 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404c785-c844-4a79-bfc9-ace78892a9f8 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] PATH_INFO: `/v3/projects/5a2ae703c7e34bd092e1e8b6dd44cd37` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:20.347998 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404c785-c844-4a79-bfc9-ace78892a9f8 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5a2ae703c7e34bd092e1e8b6dd44cd37)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:20.349883 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404c785-c844-4a79-bfc9-ace78892a9f8 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:20.388021 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:20.395971 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4404c785-c844-4a79-bfc9-ace78892a9f8 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a2ae703c7e34bd092e1e8b6dd44cd37', 'request_id': 'req-4404c785-c844-4a79-bfc9-ace78892a9f8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:20.401364 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4404c785-c844-4a79-bfc9-ace78892a9f8 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a2ae703c7e34bd092e1e8b6dd44cd37', 'request_id': 'req-4404c785-c844-4a79-bfc9-ace78892a9f8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:20.407835 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4404c785-c844-4a79-bfc9-ace78892a9f8 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a2ae703c7e34bd092e1e8b6dd44cd37', 'request_id': 'req-4404c785-c844-4a79-bfc9-ace78892a9f8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:20.420138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4404c785-c844-4a79-bfc9-ace78892a9f8 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] The token cache is being invalidate because project 5a2ae703c7e34bd092e1e8b6dd44cd37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:43:20.420415 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4404c785-c844-4a79-bfc9-ace78892a9f8 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:20.421039 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4404c785-c844-4a79-bfc9-ace78892a9f8 tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:20.422527 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1451/2889] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:43:20 2022] DELETE /identity/v3/projects/5a2ae703c7e34bd092e1e8b6dd44cd37 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:20.610756 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cb24c-43be-48d4-9949-9200ec1892ad tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:20.612561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bc5cb24c-43be-48d4-9949-9200ec1892ad tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:20.678499 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cb24c-43be-48d4-9949-9200ec1892ad tempest-VolumeQuotaClassesTest-1732095839 tempest-VolumeQuotaClassesTest-1732095839-project-admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:20.731926 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:20.732528 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:20.732798 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:20.733063 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] PATH_INFO: `/v3/users/fffb363773ed4c0cada22c7dce5638ab` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:20.734208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fffb363773ed4c0cada22c7dce5638ab)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:20.736426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:20.780221 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fffb363773ed4c0cada22c7dce5638ab'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:20.780850 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fffb363773ed4c0cada22c7dce5638ab'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:20.788143 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fffb363773ed4c0cada22c7dce5638ab'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:20.794457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fffb363773ed4c0cada22c7dce5638ab'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:20.796282 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:20.807354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fffb363773ed4c0cada22c7dce5638ab'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:20.813343 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1439/2890] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:43:20 2022] DELETE /identity/v3/users/fffb363773ed4c0cada22c7dce5638ab => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:20.817951 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b73eb-5260-4863-9be0-b39633883cdf None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:20.818292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b73eb-5260-4863-9be0-b39633883cdf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:20.818567 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b73eb-5260-4863-9be0-b39633883cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:20.837727 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-763b73eb-5260-4863-9be0-b39633883cdf None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:20.838967 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-763b73eb-5260-4863-9be0-b39633883cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:20.901472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1452/2891] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:20.911149 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:20.912542 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:20.968569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a50c3-e138-48d6-b227-6cf471928ae8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:20.972283 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695c5c5-0f23-47a2-90bb-08c8a24d2c8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:20.972872 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695c5c5-0f23-47a2-90bb-08c8a24d2c8d service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:20.973147 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695c5c5-0f23-47a2-90bb-08c8a24d2c8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:20.973417 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695c5c5-0f23-47a2-90bb-08c8a24d2c8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:20.982221 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695c5c5-0f23-47a2-90bb-08c8a24d2c8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:20.983661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695c5c5-0f23-47a2-90bb-08c8a24d2c8d service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:20.993182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1440/2892] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:21.253914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b73eb-5260-4863-9be0-b39633883cdf None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:21.255389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-763b73eb-5260-4863-9be0-b39633883cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:21.267748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b73eb-5260-4863-9be0-b39633883cdf None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:21.272536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:21.273167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:21.273450 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:21.273725 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] PATH_INFO: `/v3/projects/3499aa5ed94f492d8520354e5d5dcb43` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:21.274891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3499aa5ed94f492d8520354e5d5dcb43)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:21.276799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:21.326098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3499aa5ed94f492d8520354e5d5dcb43', 'request_id': 'req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.331440 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3499aa5ed94f492d8520354e5d5dcb43', 'request_id': 'req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.337685 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3499aa5ed94f492d8520354e5d5dcb43', 'request_id': 'req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.349975 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] The token cache is being invalidate because project 3499aa5ed94f492d8520354e5d5dcb43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:43:21.350293 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.350932 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.352460 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1453/2893] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:43:21 2022] DELETE /identity/v3/projects/3499aa5ed94f492d8520354e5d5dcb43 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:21.356487 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695c5c5-0f23-47a2-90bb-08c8a24d2c8d service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:21.366684 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695c5c5-0f23-47a2-90bb-08c8a24d2c8d service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:21.421683 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b067dd-ba2c-4929-a9f9-d474c91d7fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:21.422342 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b067dd-ba2c-4929-a9f9-d474c91d7fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:21.422630 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b067dd-ba2c-4929-a9f9-d474c91d7fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:21.422903 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b067dd-ba2c-4929-a9f9-d474c91d7fb7 admin admin] PATH_INFO: `/v3/users/db9721b21d39425989878dbae37afcd8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:21.424515 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b067dd-ba2c-4929-a9f9-d474c91d7fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db9721b21d39425989878dbae37afcd8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:21.426534 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b067dd-ba2c-4929-a9f9-d474c91d7fb7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:21.470453 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-87b067dd-ba2c-4929-a9f9-d474c91d7fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db9721b21d39425989878dbae37afcd8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.471126 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-87b067dd-ba2c-4929-a9f9-d474c91d7fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db9721b21d39425989878dbae37afcd8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.477805 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-87b067dd-ba2c-4929-a9f9-d474c91d7fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db9721b21d39425989878dbae37afcd8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.484566 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-87b067dd-ba2c-4929-a9f9-d474c91d7fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db9721b21d39425989878dbae37afcd8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.497055 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-87b067dd-ba2c-4929-a9f9-d474c91d7fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db9721b21d39425989878dbae37afcd8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.503394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1441/2894] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:43:21 2022] DELETE /identity/v3/users/db9721b21d39425989878dbae37afcd8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:21.771007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:21.772599 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:21.786715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8222cdb-2ba4-4fcf-9156-1114e7880ed8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:21.838593 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:21.850097 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:21.850748 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:21.851027 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:21.851300 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72 admin admin] PATH_INFO: `/v3/projects/ffb4b4b08ab0422bafb079ff2524e3b0` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:21.852410 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb4b4b08ab0422bafb079ff2524e3b0)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:21.854337 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:21.900808 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb4b4b08ab0422bafb079ff2524e3b0', 'request_id': 'req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.906774 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb4b4b08ab0422bafb079ff2524e3b0', 'request_id': 'req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.913490 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb4b4b08ab0422bafb079ff2524e3b0', 'request_id': 'req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.925303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72 admin admin] The token cache is being invalidate because project ffb4b4b08ab0422bafb079ff2524e3b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:43:21.925582 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.926220 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b076d793-02a3-4222-a3ed-fcfe7b2b8e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:21.928703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1454/2895] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:43:21 2022] DELETE /identity/v3/projects/ffb4b4b08ab0422bafb079ff2524e3b0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:43:21.933691 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f97dea-7e7e-41c6-939d-c6a51e87b23a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:21.934203 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f97dea-7e7e-41c6-939d-c6a51e87b23a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:21.934328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f97dea-7e7e-41c6-939d-c6a51e87b23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:21.942792 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:21.956320 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-40f97dea-7e7e-41c6-939d-c6a51e87b23a None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:21.957457 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-40f97dea-7e7e-41c6-939d-c6a51e87b23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:22.016837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1442/2896] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:22.025103 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372ec2d-8258-47dd-a021-3f8d3b28a628 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.025389 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372ec2d-8258-47dd-a021-3f8d3b28a628 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.025666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372ec2d-8258-47dd-a021-3f8d3b28a628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.045624 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6372ec2d-8258-47dd-a021-3f8d3b28a628 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:22.046864 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6372ec2d-8258-47dd-a021-3f8d3b28a628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:22.055033 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1455/2897] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:22.059202 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f97dea-7e7e-41c6-939d-c6a51e87b23a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:22.068777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f97dea-7e7e-41c6-939d-c6a51e87b23a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:22.073733 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd4b73-1fec-412d-8a57-4669bd8f61f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:22.074359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd4b73-1fec-412d-8a57-4669bd8f61f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.074638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd4b73-1fec-412d-8a57-4669bd8f61f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.074913 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fd4b73-1fec-412d-8a57-4669bd8f61f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.075557 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd4b73-1fec-412d-8a57-4669bd8f61f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:43:22.075861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd4b73-1fec-412d-8a57-4669bd8f61f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:22.077444 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fd4b73-1fec-412d-8a57-4669bd8f61f5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:22.089351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1443/2898] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:43:22.093668 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372ec2d-8258-47dd-a021-3f8d3b28a628 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:22.103363 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372ec2d-8258-47dd-a021-3f8d3b28a628 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:22.107952 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c31c27-3325-424b-829d-2dc3c86fb1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:22.108531 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c31c27-3325-424b-829d-2dc3c86fb1bc admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.108802 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c31c27-3325-424b-829d-2dc3c86fb1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.109068 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c31c27-3325-424b-829d-2dc3c86fb1bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.109688 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c31c27-3325-424b-829d-2dc3c86fb1bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:22.111640 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c31c27-3325-424b-829d-2dc3c86fb1bc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:22.138079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1456/2899] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:43:22 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:22.142354 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd4b73-1fec-412d-8a57-4669bd8f61f5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:22.152662 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fd4b73-1fec-412d-8a57-4669bd8f61f5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:22.208560 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a4772-d2be-4bc6-9372-1ca581447e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:22.209082 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a4772-d2be-4bc6-9372-1ca581447e1a admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.209363 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a4772-d2be-4bc6-9372-1ca581447e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.209633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a4772-d2be-4bc6-9372-1ca581447e1a admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.210294 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a4772-d2be-4bc6-9372-1ca581447e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:22.212218 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a4772-d2be-4bc6-9372-1ca581447e1a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:22.233366 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1444/2900] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:43:22 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:22.237600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c31c27-3325-424b-829d-2dc3c86fb1bc admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:22.248311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c31c27-3325-424b-829d-2dc3c86fb1bc admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:22.252848 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fcd40-20aa-4545-a926-af4fb35bca54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:22.253461 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fcd40-20aa-4545-a926-af4fb35bca54 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.253741 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fcd40-20aa-4545-a926-af4fb35bca54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.254008 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fcd40-20aa-4545-a926-af4fb35bca54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.254667 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fcd40-20aa-4545-a926-af4fb35bca54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:22.256241 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fcd40-20aa-4545-a926-af4fb35bca54 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:22.266191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1457/2901] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:22.270321 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a4772-d2be-4bc6-9372-1ca581447e1a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:22.281208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a4772-d2be-4bc6-9372-1ca581447e1a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:22.287695 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623b0a6-7fc6-43b8-b2f4-853ea2440b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:22.288323 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623b0a6-7fc6-43b8-b2f4-853ea2440b53 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.288600 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623b0a6-7fc6-43b8-b2f4-853ea2440b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.288935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623b0a6-7fc6-43b8-b2f4-853ea2440b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.289575 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623b0a6-7fc6-43b8-b2f4-853ea2440b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:22.291182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623b0a6-7fc6-43b8-b2f4-853ea2440b53 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:22.305860 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1445/2902] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:22.306909 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fcd40-20aa-4545-a926-af4fb35bca54 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:22.317556 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fcd40-20aa-4545-a926-af4fb35bca54 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:22.322361 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528cf6c-9914-4a2b-a1de-f44da3ef67db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:22.323008 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528cf6c-9914-4a2b-a1de-f44da3ef67db admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.323284 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528cf6c-9914-4a2b-a1de-f44da3ef67db admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.323557 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528cf6c-9914-4a2b-a1de-f44da3ef67db admin admin] PATH_INFO: `/v3/projects/c6a15766063040628d9d83e8ec4d04f9/users/fcbce1023f554710aebfac02ecd5cc06/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.339098 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528cf6c-9914-4a2b-a1de-f44da3ef67db admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a15766063040628d9d83e8ec4d04f9, user_id=fcbce1023f554710aebfac02ecd5cc06, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:22.341823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528cf6c-9914-4a2b-a1de-f44da3ef67db admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:22.355372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1458/2903] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:22 2022] PUT /identity/v3/projects/c6a15766063040628d9d83e8ec4d04f9/users/fcbce1023f554710aebfac02ecd5cc06/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:22.367026 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab687453-4ac0-40cd-94c6-01fb167075d5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.367312 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab687453-4ac0-40cd-94c6-01fb167075d5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.367598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab687453-4ac0-40cd-94c6-01fb167075d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.395556 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ab687453-4ac0-40cd-94c6-01fb167075d5 None None] MFA Rules not processed for user `fcbce1023f554710aebfac02ecd5cc06`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:22.396743 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ab687453-4ac0-40cd-94c6-01fb167075d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:22.455694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:22.459976 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1446/2904] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:22.464118 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528cf6c-9914-4a2b-a1de-f44da3ef67db admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:22.523717 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528cf6c-9914-4a2b-a1de-f44da3ef67db admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:22.528295 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98952-a2ae-4836-9ef6-dc5bf1bb9ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:22.528884 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98952-a2ae-4836-9ef6-dc5bf1bb9ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.529157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98952-a2ae-4836-9ef6-dc5bf1bb9ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.529420 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98952-a2ae-4836-9ef6-dc5bf1bb9ca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.530034 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e98952-a2ae-4836-9ef6-dc5bf1bb9ca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:22.531939 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e98952-a2ae-4836-9ef6-dc5bf1bb9ca3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:22.557161 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1459/2905] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:43:22 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:22.561313 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab687453-4ac0-40cd-94c6-01fb167075d5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:22.572195 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab687453-4ac0-40cd-94c6-01fb167075d5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:22.626658 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613baa5-7ba5-4270-97b2-524278f9be95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:22.627739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613baa5-7ba5-4270-97b2-524278f9be95 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.628011 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613baa5-7ba5-4270-97b2-524278f9be95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.628306 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613baa5-7ba5-4270-97b2-524278f9be95 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.628943 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613baa5-7ba5-4270-97b2-524278f9be95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:22.630855 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613baa5-7ba5-4270-97b2-524278f9be95 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:22.652175 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1447/2906] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:43:22 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:22.656058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98952-a2ae-4836-9ef6-dc5bf1bb9ca3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:22.661894 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:22.669610 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98952-a2ae-4836-9ef6-dc5bf1bb9ca3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:22.674960 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c4ed3-a2b7-47ea-bae2-0c4c8784aa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:22.675548 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c4ed3-a2b7-47ea-bae2-0c4c8784aa47 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.675823 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c4ed3-a2b7-47ea-bae2-0c4c8784aa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.676091 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c4ed3-a2b7-47ea-bae2-0c4c8784aa47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.676723 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c4ed3-a2b7-47ea-bae2-0c4c8784aa47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:22.678330 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c4ed3-a2b7-47ea-bae2-0c4c8784aa47 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:22.688752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1460/2907] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 27 17:43:22.693257 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613baa5-7ba5-4270-97b2-524278f9be95 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:22.702916 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613baa5-7ba5-4270-97b2-524278f9be95 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:22.708094 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd2105-fc17-4690-858c-321c39ed05b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:22.708670 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd2105-fc17-4690-858c-321c39ed05b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.708940 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd2105-fc17-4690-858c-321c39ed05b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.709201 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd2105-fc17-4690-858c-321c39ed05b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.709827 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd2105-fc17-4690-858c-321c39ed05b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:22.711483 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fd2105-fc17-4690-858c-321c39ed05b1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:22.721089 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1448/2908] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:22.725345 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c4ed3-a2b7-47ea-bae2-0c4c8784aa47 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:22.735104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c4ed3-a2b7-47ea-bae2-0c4c8784aa47 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:22.740462 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7c682-6a03-419b-9ed2-ff8a0234aca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:22.741103 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7c682-6a03-419b-9ed2-ff8a0234aca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.741652 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7c682-6a03-419b-9ed2-ff8a0234aca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.741928 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7c682-6a03-419b-9ed2-ff8a0234aca6 admin admin] PATH_INFO: `/v3/projects/70b87eca73614111b20fda34ea15fc62/users/30ea048db2a6463d89974a324a027fb1/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.757326 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7c682-6a03-419b-9ed2-ff8a0234aca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70b87eca73614111b20fda34ea15fc62, user_id=30ea048db2a6463d89974a324a027fb1, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:22.760311 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7c682-6a03-419b-9ed2-ff8a0234aca6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:22.772936 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1461/2909] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:22 2022] PUT /identity/v3/projects/70b87eca73614111b20fda34ea15fc62/users/30ea048db2a6463d89974a324a027fb1/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:22.783185 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9451d6e-12f7-4de4-8d4f-57048b65366b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.783463 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9451d6e-12f7-4de4-8d4f-57048b65366b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.783730 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9451d6e-12f7-4de4-8d4f-57048b65366b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.810945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e9451d6e-12f7-4de4-8d4f-57048b65366b None None] MFA Rules not processed for user `30ea048db2a6463d89974a324a027fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:22.812128 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e9451d6e-12f7-4de4-8d4f-57048b65366b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:22.881661 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1449/2910] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:22.886110 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7c682-6a03-419b-9ed2-ff8a0234aca6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:22.946456 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7c682-6a03-419b-9ed2-ff8a0234aca6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:22.951049 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba086d6b-79a8-4f57-a4d1-b1dd77e9122a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:22.951638 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba086d6b-79a8-4f57-a4d1-b1dd77e9122a admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:22.951914 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba086d6b-79a8-4f57-a4d1-b1dd77e9122a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:22.952182 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba086d6b-79a8-4f57-a4d1-b1dd77e9122a admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:22.952811 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba086d6b-79a8-4f57-a4d1-b1dd77e9122a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:22.954741 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba086d6b-79a8-4f57-a4d1-b1dd77e9122a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:22.980066 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1462/2911] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:43:22 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:22.984099 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9451d6e-12f7-4de4-8d4f-57048b65366b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:22.995495 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9451d6e-12f7-4de4-8d4f-57048b65366b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:23.050489 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba53e7a-9120-4bd2-9a48-7f32952dd80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:23.051078 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba53e7a-9120-4bd2-9a48-7f32952dd80f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:23.051353 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba53e7a-9120-4bd2-9a48-7f32952dd80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:23.051619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba53e7a-9120-4bd2-9a48-7f32952dd80f admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:23.052244 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba53e7a-9120-4bd2-9a48-7f32952dd80f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:23.054178 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba53e7a-9120-4bd2-9a48-7f32952dd80f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:23.069851 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:23.077219 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1450/2912] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:43:22 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:23.081365 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba086d6b-79a8-4f57-a4d1-b1dd77e9122a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:23.091378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba086d6b-79a8-4f57-a4d1-b1dd77e9122a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:23.096086 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5de17-f590-4965-a639-e06917dd2ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:23.096682 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5de17-f590-4965-a639-e06917dd2ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:23.096967 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5de17-f590-4965-a639-e06917dd2ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:23.097270 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5de17-f590-4965-a639-e06917dd2ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:23.097924 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5de17-f590-4965-a639-e06917dd2ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:23.099600 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5de17-f590-4965-a639-e06917dd2ce9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:23.112749 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1463/2913] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:23.117529 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba53e7a-9120-4bd2-9a48-7f32952dd80f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:23.128750 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba53e7a-9120-4bd2-9a48-7f32952dd80f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:23.133391 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b92dd0-1033-41a4-b768-05662420bea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:23.134031 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b92dd0-1033-41a4-b768-05662420bea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:23.134344 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b92dd0-1033-41a4-b768-05662420bea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:23.134617 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b92dd0-1033-41a4-b768-05662420bea6 admin admin] PATH_INFO: `/v3/projects/4c45dcccfb26445eb341397fe175ea5f/users/b7c0f4adebfe4ef4bf300a5a82ebb78d/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:23.148835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b92dd0-1033-41a4-b768-05662420bea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c45dcccfb26445eb341397fe175ea5f, user_id=b7c0f4adebfe4ef4bf300a5a82ebb78d, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:23.151511 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b92dd0-1033-41a4-b768-05662420bea6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:23.164036 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1451/2914] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:23 2022] PUT /identity/v3/projects/4c45dcccfb26445eb341397fe175ea5f/users/b7c0f4adebfe4ef4bf300a5a82ebb78d/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 27 17:43:23.168005 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5de17-f590-4965-a639-e06917dd2ce9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:23.178512 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5de17-f590-4965-a639-e06917dd2ce9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:23.232977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebc9601-1cec-4009-8037-04be4d444738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:23.233511 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc9601-1cec-4009-8037-04be4d444738 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:23.233799 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc9601-1cec-4009-8037-04be4d444738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:23.234057 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc9601-1cec-4009-8037-04be4d444738 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:23.234728 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc9601-1cec-4009-8037-04be4d444738 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:23.236912 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebc9601-1cec-4009-8037-04be4d444738 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:23.246921 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1464/2915] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:23.251006 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b92dd0-1033-41a4-b768-05662420bea6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:23.261095 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b92dd0-1033-41a4-b768-05662420bea6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:23.265690 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cd8ac-0d8c-4c52-a3dc-7f6fbf50e66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:23.266359 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cd8ac-0d8c-4c52-a3dc-7f6fbf50e66a admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:23.266633 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cd8ac-0d8c-4c52-a3dc-7f6fbf50e66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:23.266902 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cd8ac-0d8c-4c52-a3dc-7f6fbf50e66a admin admin] PATH_INFO: `/v3/projects/4c45dcccfb26445eb341397fe175ea5f/users/b7c0f4adebfe4ef4bf300a5a82ebb78d/roles/1dad3f1944544ab78cc0d01f38569791` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:23.268865 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cd8ac-0d8c-4c52-a3dc-7f6fbf50e66a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c45dcccfb26445eb341397fe175ea5f, user_id=b7c0f4adebfe4ef4bf300a5a82ebb78d, role_id=1dad3f1944544ab78cc0d01f38569791)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:23.271518 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cd8ac-0d8c-4c52-a3dc-7f6fbf50e66a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:23.281767 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1452/2916] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:23 2022] PUT /identity/v3/projects/4c45dcccfb26445eb341397fe175ea5f/users/b7c0f4adebfe4ef4bf300a5a82ebb78d/roles/1dad3f1944544ab78cc0d01f38569791 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:23.292299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dff5d-7884-4fb9-be64-fe1cdc236a86 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:23.292585 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dff5d-7884-4fb9-be64-fe1cdc236a86 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:23.292865 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411dff5d-7884-4fb9-be64-fe1cdc236a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:23.322215 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-411dff5d-7884-4fb9-be64-fe1cdc236a86 None None] MFA Rules not processed for user `b7c0f4adebfe4ef4bf300a5a82ebb78d`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:23.323433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-411dff5d-7884-4fb9-be64-fe1cdc236a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:23.349735 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:23.392062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1465/2917] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:23.403380 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cd8ac-0d8c-4c52-a3dc-7f6fbf50e66a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:23.405073 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-017cd8ac-0d8c-4c52-a3dc-7f6fbf50e66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:23.460935 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cd8ac-0d8c-4c52-a3dc-7f6fbf50e66a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:23.464536 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecde12-1527-4440-b869-c577f92e7005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:23.465141 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecde12-1527-4440-b869-c577f92e7005 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:23.465394 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecde12-1527-4440-b869-c577f92e7005 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:23.465662 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eecde12-1527-4440-b869-c577f92e7005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:23.521430 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecde12-1527-4440-b869-c577f92e7005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:23.522894 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eecde12-1527-4440-b869-c577f92e7005 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:23.531676 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1453/2918] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:25.036699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411dff5d-7884-4fb9-be64-fe1cdc236a86 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:25.038138 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-411dff5d-7884-4fb9-be64-fe1cdc236a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:25.049768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411dff5d-7884-4fb9-be64-fe1cdc236a86 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:25.053288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0a24b-b604-401e-94ae-3e9ec9c59d12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:25.053882 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a24b-b604-401e-94ae-3e9ec9c59d12 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:25.054191 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a24b-b604-401e-94ae-3e9ec9c59d12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:25.054466 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a24b-b604-401e-94ae-3e9ec9c59d12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:25.111314 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0a24b-b604-401e-94ae-3e9ec9c59d12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:25.112744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0a24b-b604-401e-94ae-3e9ec9c59d12 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:25.122012 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1466/2919] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:25.651729 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecde12-1527-4440-b869-c577f92e7005 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:25.660335 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eecde12-1527-4440-b869-c577f92e7005 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:25.664020 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c1b00-160e-403a-91e4-ff34ebe3a4ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:25.664638 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c1b00-160e-403a-91e4-ff34ebe3a4ce service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:25.664912 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c1b00-160e-403a-91e4-ff34ebe3a4ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:25.665208 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c1b00-160e-403a-91e4-ff34ebe3a4ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:25.674086 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c1b00-160e-403a-91e4-ff34ebe3a4ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:25.675537 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c1b00-160e-403a-91e4-ff34ebe3a4ce service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:25.684777 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1454/2920] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:29.550367 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0a24b-b604-401e-94ae-3e9ec9c59d12 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:29.624079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0a24b-b604-401e-94ae-3e9ec9c59d12 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:29.628536 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:29.629169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:29.629442 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:29.629705 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] PATH_INFO: `/v3/users/fcbce1023f554710aebfac02ecd5cc06` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:29.630810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcbce1023f554710aebfac02ecd5cc06)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:29.632702 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:29.679062 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcbce1023f554710aebfac02ecd5cc06'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:29.685785 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcbce1023f554710aebfac02ecd5cc06'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:29.686459 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcbce1023f554710aebfac02ecd5cc06'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:29.694406 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcbce1023f554710aebfac02ecd5cc06'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:29.695516 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:29.709096 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcbce1023f554710aebfac02ecd5cc06'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:29.714859 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1467/2921] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:43:29 2022] DELETE /identity/v3/users/fcbce1023f554710aebfac02ecd5cc06 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:29.719318 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23138fc-dfd9-4f8d-85ab-de409c347f9c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:29.719815 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23138fc-dfd9-4f8d-85ab-de409c347f9c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:29.719922 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23138fc-dfd9-4f8d-85ab-de409c347f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:29.739470 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e23138fc-dfd9-4f8d-85ab-de409c347f9c None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:29.740681 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e23138fc-dfd9-4f8d-85ab-de409c347f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:29.798184 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1455/2922] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:29.807783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:29.809217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:29.873977 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b2830-f5bb-4ee3-931b-862ed2402833 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:29.878842 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dad6ce-6748-4b5e-95fe-20d38b289aa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:29.880087 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dad6ce-6748-4b5e-95fe-20d38b289aa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:29.880360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dad6ce-6748-4b5e-95fe-20d38b289aa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:29.880625 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dad6ce-6748-4b5e-95fe-20d38b289aa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:29.889866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dad6ce-6748-4b5e-95fe-20d38b289aa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:29.891340 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dad6ce-6748-4b5e-95fe-20d38b289aa4 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:29.904726 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1468/2923] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:30.181932 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23138fc-dfd9-4f8d-85ab-de409c347f9c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:30.183984 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e23138fc-dfd9-4f8d-85ab-de409c347f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:30.196824 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:30.203464 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23138fc-dfd9-4f8d-85ab-de409c347f9c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:30.210834 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:30.211582 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:30.211989 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:30.212371 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] PATH_INFO: `/v3/projects/c6a15766063040628d9d83e8ec4d04f9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:30.213569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a15766063040628d9d83e8ec4d04f9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:30.215503 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:30.262594 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a15766063040628d9d83e8ec4d04f9', 'request_id': 'req-011785af-134d-4170-b0a9-0430f864b21f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.268317 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a15766063040628d9d83e8ec4d04f9', 'request_id': 'req-011785af-134d-4170-b0a9-0430f864b21f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.274463 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a15766063040628d9d83e8ec4d04f9', 'request_id': 'req-011785af-134d-4170-b0a9-0430f864b21f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.286623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] The token cache is being invalidate because project c6a15766063040628d9d83e8ec4d04f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:43:30.286936 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.287555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.289135 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1456/2924] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:43:30 2022] DELETE /identity/v3/projects/c6a15766063040628d9d83e8ec4d04f9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:30.293315 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dad6ce-6748-4b5e-95fe-20d38b289aa4 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:30.303542 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dad6ce-6748-4b5e-95fe-20d38b289aa4 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:30.359472 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:30.360095 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:30.360372 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:30.360643 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] PATH_INFO: `/v3/users/30ea048db2a6463d89974a324a027fb1` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:30.361752 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ea048db2a6463d89974a324a027fb1)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:30.364246 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:30.409492 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ea048db2a6463d89974a324a027fb1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.416292 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ea048db2a6463d89974a324a027fb1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.416941 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ea048db2a6463d89974a324a027fb1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.424050 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ea048db2a6463d89974a324a027fb1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.436724 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ea048db2a6463d89974a324a027fb1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.443335 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1469/2925] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:43:30 2022] DELETE /identity/v3/users/30ea048db2a6463d89974a324a027fb1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:30.708304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:30.709791 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:30.723561 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011785af-134d-4170-b0a9-0430f864b21f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:30.781137 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:30.781759 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:30.782037 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:30.782347 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] PATH_INFO: `/v3/projects/70b87eca73614111b20fda34ea15fc62` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:30.783449 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70b87eca73614111b20fda34ea15fc62)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:30.785877 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:30.834557 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70b87eca73614111b20fda34ea15fc62', 'request_id': 'req-cc08ea4c-bf14-40ac-add9-d9582ccc4310'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.839975 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70b87eca73614111b20fda34ea15fc62', 'request_id': 'req-cc08ea4c-bf14-40ac-add9-d9582ccc4310'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.846012 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70b87eca73614111b20fda34ea15fc62', 'request_id': 'req-cc08ea4c-bf14-40ac-add9-d9582ccc4310'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.858956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] The token cache is being invalidate because project 70b87eca73614111b20fda34ea15fc62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:43:30.859241 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.859847 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.861322 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1457/2926] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:43:30 2022] DELETE /identity/v3/projects/70b87eca73614111b20fda34ea15fc62 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:30.865988 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:30.867688 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:30.875587 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:30.882159 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc999991-8d37-46b7-ad2f-d2f82186a98b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:30.938149 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49f063-a76d-435f-bc76-4bdbcac51ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:30.938765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49f063-a76d-435f-bc76-4bdbcac51ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:30.939046 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49f063-a76d-435f-bc76-4bdbcac51ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:30.939320 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49f063-a76d-435f-bc76-4bdbcac51ff1 admin admin] PATH_INFO: `/v3/users/b7c0f4adebfe4ef4bf300a5a82ebb78d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:30.940433 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49f063-a76d-435f-bc76-4bdbcac51ff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7c0f4adebfe4ef4bf300a5a82ebb78d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:30.942428 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49f063-a76d-435f-bc76-4bdbcac51ff1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:30.987646 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2b49f063-a76d-435f-bc76-4bdbcac51ff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7c0f4adebfe4ef4bf300a5a82ebb78d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.994703 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2b49f063-a76d-435f-bc76-4bdbcac51ff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7c0f4adebfe4ef4bf300a5a82ebb78d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:30.995357 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2b49f063-a76d-435f-bc76-4bdbcac51ff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7c0f4adebfe4ef4bf300a5a82ebb78d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:31.002678 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2b49f063-a76d-435f-bc76-4bdbcac51ff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7c0f4adebfe4ef4bf300a5a82ebb78d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:31.014927 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2b49f063-a76d-435f-bc76-4bdbcac51ff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7c0f4adebfe4ef4bf300a5a82ebb78d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:31.021205 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1470/2927] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:43:30 2022] DELETE /identity/v3/users/b7c0f4adebfe4ef4bf300a5a82ebb78d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:31.287963 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:31.289468 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:31.305284 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08ea4c-bf14-40ac-add9-d9582ccc4310 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:31.313844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:31.363813 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcead46-a954-4013-abb2-8cc70ab5acfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:31.364431 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcead46-a954-4013-abb2-8cc70ab5acfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:31.364705 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcead46-a954-4013-abb2-8cc70ab5acfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:31.364972 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcead46-a954-4013-abb2-8cc70ab5acfd admin admin] PATH_INFO: `/v3/projects/4c45dcccfb26445eb341397fe175ea5f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:31.366109 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcead46-a954-4013-abb2-8cc70ab5acfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c45dcccfb26445eb341397fe175ea5f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:31.368260 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcead46-a954-4013-abb2-8cc70ab5acfd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:31.412945 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4fcead46-a954-4013-abb2-8cc70ab5acfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c45dcccfb26445eb341397fe175ea5f', 'request_id': 'req-4fcead46-a954-4013-abb2-8cc70ab5acfd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:31.418293 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4fcead46-a954-4013-abb2-8cc70ab5acfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c45dcccfb26445eb341397fe175ea5f', 'request_id': 'req-4fcead46-a954-4013-abb2-8cc70ab5acfd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:31.424357 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4fcead46-a954-4013-abb2-8cc70ab5acfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c45dcccfb26445eb341397fe175ea5f', 'request_id': 'req-4fcead46-a954-4013-abb2-8cc70ab5acfd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:31.436090 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4fcead46-a954-4013-abb2-8cc70ab5acfd admin admin] The token cache is being invalidate because project 4c45dcccfb26445eb341397fe175ea5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:43:31.436624 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4fcead46-a954-4013-abb2-8cc70ab5acfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:31.437266 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4fcead46-a954-4013-abb2-8cc70ab5acfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:31.438839 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1458/2928] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:43:31 2022] DELETE /identity/v3/projects/4c45dcccfb26445eb341397fe175ea5f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:31.444386 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8c5c0-97d5-4444-99bc-04379096fdca None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:31.444669 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8c5c0-97d5-4444-99bc-04379096fdca None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:31.444942 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8c5c0-97d5-4444-99bc-04379096fdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:31.464768 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3bb8c5c0-97d5-4444-99bc-04379096fdca None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:31.465961 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3bb8c5c0-97d5-4444-99bc-04379096fdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:31.523324 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1471/2929] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:31.531619 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6dde9-9bb1-46b5-bfdc-b6fb6190e742 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:31.531835 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6dde9-9bb1-46b5-bfdc-b6fb6190e742 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:31.532101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6dde9-9bb1-46b5-bfdc-b6fb6190e742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:31.551199 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2cc6dde9-9bb1-46b5-bfdc-b6fb6190e742 None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:31.552410 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2cc6dde9-9bb1-46b5-bfdc-b6fb6190e742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:31.560066 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1459/2930] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:31.564160 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8c5c0-97d5-4444-99bc-04379096fdca None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:31.574735 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8c5c0-97d5-4444-99bc-04379096fdca None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:31.579299 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b0798-193e-4f5e-8901-6db09d83f1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:31.579892 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b0798-193e-4f5e-8901-6db09d83f1eb admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:31.580168 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b0798-193e-4f5e-8901-6db09d83f1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:31.580439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b0798-193e-4f5e-8901-6db09d83f1eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:31.581107 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b0798-193e-4f5e-8901-6db09d83f1eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 27 17:43:31.581394 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b0798-193e-4f5e-8901-6db09d83f1eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:31.583564 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b0798-193e-4f5e-8901-6db09d83f1eb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:31.594897 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1472/2931] 10.222.0.55 () {58 vars in 1153 bytes} [Tue Sep 27 17:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 27 17:43:31.599048 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6dde9-9bb1-46b5-bfdc-b6fb6190e742 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:31.608153 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6dde9-9bb1-46b5-bfdc-b6fb6190e742 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:31.612721 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76760c3d-ff88-4416-9c91-ba10df4bb652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:31.613303 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76760c3d-ff88-4416-9c91-ba10df4bb652 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:31.613577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76760c3d-ff88-4416-9c91-ba10df4bb652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:31.613844 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76760c3d-ff88-4416-9c91-ba10df4bb652 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:31.614504 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76760c3d-ff88-4416-9c91-ba10df4bb652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:31.616369 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76760c3d-ff88-4416-9c91-ba10df4bb652 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:31.645152 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1460/2932] 10.222.0.55 () {60 vars in 1153 bytes} [Tue Sep 27 17:43:31 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:31.648007 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b0798-193e-4f5e-8901-6db09d83f1eb admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:31.657872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b0798-193e-4f5e-8901-6db09d83f1eb admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:31.662515 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:31.716296 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb2f7d-057b-4013-9938-16752c64ea40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:31.716892 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb2f7d-057b-4013-9938-16752c64ea40 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:31.717167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb2f7d-057b-4013-9938-16752c64ea40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:31.717438 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb2f7d-057b-4013-9938-16752c64ea40 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:31.718088 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb2f7d-057b-4013-9938-16752c64ea40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:31.720003 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb2f7d-057b-4013-9938-16752c64ea40 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:31.740872 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1473/2933] 10.222.0.55 () {60 vars in 1144 bytes} [Tue Sep 27 17:43:31 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 27 17:43:31.745304 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76760c3d-ff88-4416-9c91-ba10df4bb652 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:31.756210 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76760c3d-ff88-4416-9c91-ba10df4bb652 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:31.760717 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2ea2a-3518-4b9b-a491-0930567e32c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:31.761295 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2ea2a-3518-4b9b-a491-0930567e32c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:31.761569 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2ea2a-3518-4b9b-a491-0930567e32c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:31.761861 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2ea2a-3518-4b9b-a491-0930567e32c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:31.762555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2ea2a-3518-4b9b-a491-0930567e32c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:31.764120 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2ea2a-3518-4b9b-a491-0930567e32c6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:31.774084 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1461/2934] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:31.777845 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb2f7d-057b-4013-9938-16752c64ea40 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:31.787765 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb2f7d-057b-4013-9938-16752c64ea40 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:31.792308 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7adb89-5ab1-4bd9-8dc1-22fd0eac0863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:31.792891 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7adb89-5ab1-4bd9-8dc1-22fd0eac0863 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:31.793169 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7adb89-5ab1-4bd9-8dc1-22fd0eac0863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:31.793439 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7adb89-5ab1-4bd9-8dc1-22fd0eac0863 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:31.794099 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7adb89-5ab1-4bd9-8dc1-22fd0eac0863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:31.795693 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7adb89-5ab1-4bd9-8dc1-22fd0eac0863 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:31.806699 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1474/2935] 10.222.0.55 () {58 vars in 1122 bytes} [Tue Sep 27 17:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 27 17:43:31.810574 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2ea2a-3518-4b9b-a491-0930567e32c6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:31.820915 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2ea2a-3518-4b9b-a491-0930567e32c6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:31.825455 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909fbe3-c275-4085-a919-cb84208479f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:31.826100 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909fbe3-c275-4085-a919-cb84208479f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:31.826579 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909fbe3-c275-4085-a919-cb84208479f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:31.826708 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909fbe3-c275-4085-a919-cb84208479f8 admin admin] PATH_INFO: `/v3/projects/aad3fbebc440463194bf8cf5ac707f13/users/753045042fbe4a19b40ccd30bec24cdc/roles/c135afb53eaf43d9a99716160f0ffd25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:31.841404 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909fbe3-c275-4085-a919-cb84208479f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aad3fbebc440463194bf8cf5ac707f13, user_id=753045042fbe4a19b40ccd30bec24cdc, role_id=c135afb53eaf43d9a99716160f0ffd25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:31.844056 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909fbe3-c275-4085-a919-cb84208479f8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:31.857160 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1462/2936] 10.222.0.55 () {60 vars in 1483 bytes} [Tue Sep 27 17:43:31 2022] PUT /identity/v3/projects/aad3fbebc440463194bf8cf5ac707f13/users/753045042fbe4a19b40ccd30bec24cdc/roles/c135afb53eaf43d9a99716160f0ffd25 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:31.867671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99d8f1-275f-4d62-8c41-c4b2870b9af2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:31.868157 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99d8f1-275f-4d62-8c41-c4b2870b9af2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:31.868303 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99d8f1-275f-4d62-8c41-c4b2870b9af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:31.896632 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-cb99d8f1-275f-4d62-8c41-c4b2870b9af2 None None] MFA Rules not processed for user `753045042fbe4a19b40ccd30bec24cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:31.897866 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cb99d8f1-275f-4d62-8c41-c4b2870b9af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:31.960793 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1475/2937] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:31.971242 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909fbe3-c275-4085-a919-cb84208479f8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:31.973473 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b909fbe3-c275-4085-a919-cb84208479f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:32.018509 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:32.032101 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909fbe3-c275-4085-a919-cb84208479f8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:32.036345 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16887b3-f7a9-4829-bbca-68c1a59b80df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '528981e41f0b436b82989d62c713d993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:32.036939 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16887b3-f7a9-4829-bbca-68c1a59b80df service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:32.037217 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16887b3-f7a9-4829-bbca-68c1a59b80df service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:32.037488 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16887b3-f7a9-4829-bbca-68c1a59b80df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:32.045837 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16887b3-f7a9-4829-bbca-68c1a59b80df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:32.047274 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16887b3-f7a9-4829-bbca-68c1a59b80df service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:32.055998 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1463/2938] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:33.422079 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cef9d9-d4a6-40db-b426-22b2054b471d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:33.422609 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cef9d9-d4a6-40db-b426-22b2054b471d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:33.422731 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cef9d9-d4a6-40db-b426-22b2054b471d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:33.440922 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-78cef9d9-d4a6-40db-b426-22b2054b471d None None] MFA Rules not processed for user `753045042fbe4a19b40ccd30bec24cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:33.442104 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-78cef9d9-d4a6-40db-b426-22b2054b471d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:33.448744 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1476/2939] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:42.052328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:43.436217 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:51.850018 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335a957-6dab-4095-a153-86ecff14efda None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:51.850596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335a957-6dab-4095-a153-86ecff14efda None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:51.850596 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335a957-6dab-4095-a153-86ecff14efda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:51.880901 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e335a957-6dab-4095-a153-86ecff14efda None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:51.882110 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e335a957-6dab-4095-a153-86ecff14efda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:51.939580 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1464/2940] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:43:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:51.986043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cef9d9-d4a6-40db-b426-22b2054b471d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:51.987560 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-78cef9d9-d4a6-40db-b426-22b2054b471d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:52.054279 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cef9d9-d4a6-40db-b426-22b2054b471d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:52.059177 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df53fe-3bc1-4db6-aca7-94feb956d057 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:52.059794 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df53fe-3bc1-4db6-aca7-94feb956d057 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:52.060075 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df53fe-3bc1-4db6-aca7-94feb956d057 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:52.060344 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df53fe-3bc1-4db6-aca7-94feb956d057 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:52.068109 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df53fe-3bc1-4db6-aca7-94feb956d057 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:52.069666 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df53fe-3bc1-4db6-aca7-94feb956d057 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:52.077407 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1477/2941] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:43:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:43:52.581694 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4751d51-9da4-4599-a93e-3a469b55e1c8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:52.581982 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4751d51-9da4-4599-a93e-3a469b55e1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:52.582291 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4751d51-9da4-4599-a93e-3a469b55e1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:52.601706 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d4751d51-9da4-4599-a93e-3a469b55e1c8 None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:52.602937 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d4751d51-9da4-4599-a93e-3a469b55e1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:52.610007 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1465/2942] 10.222.0.55 () {58 vars in 967 bytes} [Tue Sep 27 17:43:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:52.622117 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df53fe-3bc1-4db6-aca7-94feb956d057 service swift] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:52.632934 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df53fe-3bc1-4db6-aca7-94feb956d057 service swift] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:52.637850 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18167022-2ac3-4e65-af1a-923012f4b14c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:52.638485 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18167022-2ac3-4e65-af1a-923012f4b14c service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:52.638755 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18167022-2ac3-4e65-af1a-923012f4b14c service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:52.639020 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18167022-2ac3-4e65-af1a-923012f4b14c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:52.647366 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18167022-2ac3-4e65-af1a-923012f4b14c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:52.648911 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18167022-2ac3-4e65-af1a-923012f4b14c service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:52.656657 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1478/2943] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:43:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:43:52.878051 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3990b0-40f8-46d7-8a02-a49605843a87 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:52.878552 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3990b0-40f8-46d7-8a02-a49605843a87 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:52.878663 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3990b0-40f8-46d7-8a02-a49605843a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:52.897845 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4d3990b0-40f8-46d7-8a02-a49605843a87 None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:52.899129 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4d3990b0-40f8-46d7-8a02-a49605843a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:52.906131 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1466/2944] 10.222.0.55 () {58 vars in 1009 bytes} [Tue Sep 27 17:43:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:52.917671 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18167022-2ac3-4e65-af1a-923012f4b14c service swift] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:52.928525 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18167022-2ac3-4e65-af1a-923012f4b14c service swift] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:52.933544 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f5511-a17a-4d25-9f67-8030325c8fbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:52.934167 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f5511-a17a-4d25-9f67-8030325c8fbf service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:52.934445 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f5511-a17a-4d25-9f67-8030325c8fbf service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:52.934715 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f5511-a17a-4d25-9f67-8030325c8fbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:52.943240 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f5511-a17a-4d25-9f67-8030325c8fbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:52.944821 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f5511-a17a-4d25-9f67-8030325c8fbf service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:52.953402 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1479/2945] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:43:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:43:53.271367 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdc2a9-03cc-454a-ae08-1c904d9880a0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:53.271678 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdc2a9-03cc-454a-ae08-1c904d9880a0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:53.271950 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdc2a9-03cc-454a-ae08-1c904d9880a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:53.292104 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b1fdc2a9-03cc-454a-ae08-1c904d9880a0 None None] MFA Rules not processed for user `c8c3fcd3789c40c88a4c32fff2678fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:53.293299 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b1fdc2a9-03cc-454a-ae08-1c904d9880a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:53.300333 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1467/2946] 10.222.0.55 () {58 vars in 967 bytes} [Tue Sep 27 17:43:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 27 17:43:53.313068 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f5511-a17a-4d25-9f67-8030325c8fbf service swift] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:53.323847 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f5511-a17a-4d25-9f67-8030325c8fbf service swift] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:53.328733 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8415a-2fae-4b79-8e37-cbbfce4e2a31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39073006a0054b56bb1bba46d2d0768e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:53.329378 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8415a-2fae-4b79-8e37-cbbfce4e2a31 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:53.329656 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8415a-2fae-4b79-8e37-cbbfce4e2a31 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:53.329920 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8415a-2fae-4b79-8e37-cbbfce4e2a31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:53.337741 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8415a-2fae-4b79-8e37-cbbfce4e2a31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:53.339356 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8415a-2fae-4b79-8e37-cbbfce4e2a31 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:53.347043 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1480/2947] 10.222.0.55 () {60 vars in 1384 bytes} [Tue Sep 27 17:43:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 27 17:43:55.416577 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdc2a9-03cc-454a-ae08-1c904d9880a0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:55.477182 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdc2a9-03cc-454a-ae08-1c904d9880a0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:55.481667 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:55.482310 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:55.482588 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:55.482859 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] PATH_INFO: `/v3/users/753045042fbe4a19b40ccd30bec24cdc` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:55.483981 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] RBAC: Authorizing `identity:delete_user(user_id=753045042fbe4a19b40ccd30bec24cdc)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:55.485887 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:55.530956 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '753045042fbe4a19b40ccd30bec24cdc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:55.531597 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '753045042fbe4a19b40ccd30bec24cdc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:55.538471 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '753045042fbe4a19b40ccd30bec24cdc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:55.545659 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '753045042fbe4a19b40ccd30bec24cdc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:55.556725 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:55.557968 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '753045042fbe4a19b40ccd30bec24cdc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:55.564699 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1468/2948] 10.222.0.55 () {58 vars in 1224 bytes} [Tue Sep 27 17:43:55 2022] DELETE /identity/v3/users/753045042fbe4a19b40ccd30bec24cdc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:43:55.570024 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0e9f2-0697-43fc-b121-4e4fc5a3318a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:55.570549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0e9f2-0697-43fc-b121-4e4fc5a3318a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:55.570549 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0e9f2-0697-43fc-b121-4e4fc5a3318a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:55.589588 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-10e0e9f2-0697-43fc-b121-4e4fc5a3318a None None] MFA Rules not processed for user `60f027a86a574cfdbd746a543980fbdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 27 17:43:55.590810 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-10e0e9f2-0697-43fc-b121-4e4fc5a3318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:55.647870 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1481/2949] 10.222.0.55 () {58 vars in 958 bytes} [Tue Sep 27 17:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 27 17:43:55.657680 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:55.659277 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:55.715623 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a1b23-e09a-4ea4-b245-17dbe081606b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:55.719285 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca4c2c-dc87-415a-b3ae-7e322882a909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d906452199844149966c9c86db4684e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:55.719875 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca4c2c-dc87-415a-b3ae-7e322882a909 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:55.720154 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca4c2c-dc87-415a-b3ae-7e322882a909 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:55.720426 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca4c2c-dc87-415a-b3ae-7e322882a909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:55.729276 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca4c2c-dc87-415a-b3ae-7e322882a909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:55.730739 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca4c2c-dc87-415a-b3ae-7e322882a909 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:55.740719 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1469/2950] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:43:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:43:55.913606 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0e9f2-0697-43fc-b121-4e4fc5a3318a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:43:55.915058 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-10e0e9f2-0697-43fc-b121-4e4fc5a3318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:43:55.927700 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0e9f2-0697-43fc-b121-4e4fc5a3318a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:43:55.932360 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66378fae-ea2a-4d31-8360-0deaa4064664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f027a86a574cfdbd746a543980fbdf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c93f41c1d2243bfbbfb547f2005e999', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:43:55.932986 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66378fae-ea2a-4d31-8360-0deaa4064664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:43:55.933259 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66378fae-ea2a-4d31-8360-0deaa4064664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:43:55.933528 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66378fae-ea2a-4d31-8360-0deaa4064664 admin admin] PATH_INFO: `/v3/projects/aad3fbebc440463194bf8cf5ac707f13` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:43:55.934661 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66378fae-ea2a-4d31-8360-0deaa4064664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aad3fbebc440463194bf8cf5ac707f13)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:43:55.936562 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66378fae-ea2a-4d31-8360-0deaa4064664 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:43:55.982518 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:43:55.985365 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-66378fae-ea2a-4d31-8360-0deaa4064664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aad3fbebc440463194bf8cf5ac707f13', 'request_id': 'req-66378fae-ea2a-4d31-8360-0deaa4064664'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:55.992288 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-66378fae-ea2a-4d31-8360-0deaa4064664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aad3fbebc440463194bf8cf5ac707f13', 'request_id': 'req-66378fae-ea2a-4d31-8360-0deaa4064664'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:55.998783 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-66378fae-ea2a-4d31-8360-0deaa4064664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aad3fbebc440463194bf8cf5ac707f13', 'request_id': 'req-66378fae-ea2a-4d31-8360-0deaa4064664'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:56.011635 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-66378fae-ea2a-4d31-8360-0deaa4064664 admin admin] The token cache is being invalidate because project aad3fbebc440463194bf8cf5ac707f13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 27 17:43:56.011924 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-66378fae-ea2a-4d31-8360-0deaa4064664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:56.012546 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-66378fae-ea2a-4d31-8360-0deaa4064664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 27 17:43:56.014044 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1482/2951] 10.222.0.55 () {58 vars in 1233 bytes} [Tue Sep 27 17:43:55 2022] DELETE /identity/v3/projects/aad3fbebc440463194bf8cf5ac707f13 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 27 17:44:05.735928 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=28 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:44:06.008651 dv-cinder-849143-8-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 17:47:29.764790 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca4c2c-dc87-415a-b3ae-7e322882a909 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 27 17:47:29.766214 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3aca4c2c-dc87-415a-b3ae-7e322882a909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:47:29.833479 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca4c2c-dc87-415a-b3ae-7e322882a909 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 27 17:47:29.891555 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847d726-c475-44eb-ae31-4cd991dcc615 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b16c430b870424c8d366c9a2959003e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bb1f2574a4344711a56722bca143ae6c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 27 17:47:29.892057 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847d726-c475-44eb-ae31-4cd991dcc615 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 27 17:47:29.892328 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847d726-c475-44eb-ae31-4cd991dcc615 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 27 17:47:29.892598 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847d726-c475-44eb-ae31-4cd991dcc615 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 27 17:47:29.894353 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b847d726-c475-44eb-ae31-4cd991dcc615 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 27 17:47:29.905351 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847d726-c475-44eb-ae31-4cd991dcc615 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 27 17:47:29.906953 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847d726-c475-44eb-ae31-4cd991dcc615 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 27 17:47:29.917288 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1470/2952] 10.222.0.55 () {60 vars in 1365 bytes} [Tue Sep 27 17:47:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 27 17:47:39.912573 dv-cinder-849143-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=58 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}